Jul 28 14:16:44.311983 ip-10-0-141-50 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 14:16:44.311994 ip-10-0-141-50 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 14:16:44.312000 ip-10-0-141-50 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 14:16:44.312263 ip-10-0-141-50 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 14:16:54.500267 ip-10-0-141-50 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 14:16:54.500283 ip-10-0-141-50 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 90f9b2d4dd96440897c80786230ddc81 --
Jul 28 14:18:49.619482 ip-10-0-141-50 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 14:18:49.938538 ip-10-0-141-50 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 14:18:49.938538 ip-10-0-141-50 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 14:18:49.938538 ip-10-0-141-50 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 14:18:49.938538 ip-10-0-141-50 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 14:18:49.939143 ip-10-0-141-50 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 14:18:49.939332 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.939206    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 14:18:49.943201 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943186    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:49.943201 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943201    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943204    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943208    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943223    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943226    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943229    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943232    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943235    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943238    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943240    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943243    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943246    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943248    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943251    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943254    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943256    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943259    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943262    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943265    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943268    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:49.943274 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943270    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943273    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943275    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943278    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943281    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943284    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943287    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943290    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943292    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943295    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943297    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943300    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943303    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943305    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943308    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943311    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943313    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943316    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943319    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943321    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:49.943752 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943324    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943327    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943329    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943332    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943334    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943337    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943339    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943342    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943344    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943347    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943349    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943352    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943354    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943357    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943360    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943363    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943366    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943368    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943373    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:49.944260 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943377    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943380    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943383    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943386    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943389    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943391    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943394    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943396    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943399    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943401    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943404    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943407    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943409    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943412    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943414    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943417    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943419    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943422    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943424    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943427    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:49.944716 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943431    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943434    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943438    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943442    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943445    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943448    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943451    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943455    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943458    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943462    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943465    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943468    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943471    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943473    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943476    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943479    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943481    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943484    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.943486    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944173    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:49.945196 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944179    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944181    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944184    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944187    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944190    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944193    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944196    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944198    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944201    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944204    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944207    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944209    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944228    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944230    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944233    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944236    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944238    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944240    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944243    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944246    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:49.945682 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944249    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944252    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944255    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944257    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944260    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944263    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944266    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944268    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944272    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944276    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944279    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944281    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944284    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944286    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944289    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944292    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944294    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944297    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944299    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944302    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:49.946174 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944304    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944307    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944309    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944312    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944314    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944317    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944319    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944322    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944324    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944327    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944331    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944336    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944339    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944343    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944346    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944350    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944352    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944355    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944358    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:49.946671 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944361    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944364    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944366    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944369    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944372    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944375    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944378    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944380    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944383    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944385    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944388    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944391    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944393    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944396    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944398    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944402    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944405    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944407    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944410    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944412    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:49.947140 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944415    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944417    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944420    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944422    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944426    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944429    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944431    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944434    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944437    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944440    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944442    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944445    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944447    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944450    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944452    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944455    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944457    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944460    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944462    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:49.947628 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944824    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944829    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944832    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944834    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944837    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944840    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944842    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944845    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944848    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944850    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944853    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944855    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944858    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944861    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944863    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944866    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944869    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944872    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944875    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:49.948103 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944879    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944883    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944885    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944888    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944891    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944894    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944896    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944899    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944901    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944904    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944906    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944909    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944912    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944914    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944917    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944919    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944922    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944924    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944927    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944930    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:49.948568 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944932    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944937    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944941    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944944    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944947    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944949    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944952    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944955    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944958    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944960    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944963    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944966    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944969    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944971    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944974    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944977    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944980    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944983    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944985    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944988    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:49.949074 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944990    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944992    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944995    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.944997    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945000    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945002    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945005    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945007    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945010    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945012    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945015    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945017    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945020    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945022    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945025    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945027    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945030    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945032    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945035    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945037    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:49.949612 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945040    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945042    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945045    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945048    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945050    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945053    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945055    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945058    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945061    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945063    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945066    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945069    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945072    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945075    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945078    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945081    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945084    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945086    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945089    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945091    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:49.950092 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945163    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945172    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945181    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945188    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945194    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945199    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945206    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945223    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945226    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945230    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945234    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945237    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945240    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945244    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945247    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945250    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945253    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945256    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945261    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945264    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945267    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945271    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945274    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945278    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 14:18:49.950591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945281    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945284    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945287    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945290    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945293    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945296    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945299    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945302    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945306    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945309    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945312    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945315    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945318    2579 flags.go:64] FLAG: --enable-server="true"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945321    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945325    2579 flags.go:64] FLAG: --event-burst="100"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945328    2579 flags.go:64] FLAG: --event-qps="50"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945331    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945334    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945340    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945343    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945346    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945349    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945354    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945357    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945360    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 14:18:49.951186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945363    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945366    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945369    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945371    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945374    2579 flags.go:64] FLAG: --feature-gates=""
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945378    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945381    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945385    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945388    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945391    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945393    2579 flags.go:64] FLAG: --help="false"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945396    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-141-50.ec2.internal"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945400    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945403    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945406    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945409    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945412    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945415    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945418    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945421    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945423    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945427    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945430    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945432    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 28 14:18:49.951880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945435    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945439    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945442    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945445    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945448    2579 flags.go:64] FLAG: --lock-file=""
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945452    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945455    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945458    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945463    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945466    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945469    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945472    2579 flags.go:64] FLAG: --logging-format="text"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945475    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945478    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945481    2579 flags.go:64] FLAG: --manifest-url=""
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945484    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945488    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945491    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945495    2579 flags.go:64] FLAG: --max-pods="110"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945498    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945501    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945504    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945507    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945510    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945513    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 14:18:49.952478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945516    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945523    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945526    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945529    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945532    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945535    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945541    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945543    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945547    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945551    2579 flags.go:64] FLAG: --port="10250"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945554    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945557    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ef719b9561d3306a"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945561    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945565    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945568    2579 flags.go:64] FLAG: --register-node="true"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945571    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945574    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945577    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945580    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945583    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945586    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945589    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945592    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945595    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945598    2579 flags.go:64] FLAG: --runonce="false"
Jul 28 14:18:49.953099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945601    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945604    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945607    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945610    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945612    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945615    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945618    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945621    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945624    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945626    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945629    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945632    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945635    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945638    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945641    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945646    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945652    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945655    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945659    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945663    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945666    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945669    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945672    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945675    2579 flags.go:64] FLAG: --v="2"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945678    2579 flags.go:64] FLAG: --version="false"
Jul 28 14:18:49.953731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945683    2579 flags.go:64] FLAG: --vmodule=""
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945687    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.945690    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945806    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945810    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945814    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945818    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945821    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945824    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945828    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945830    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945833    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945836    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945839    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945842    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945844    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945847    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945850    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945853    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945856    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:49.954374 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945858    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945861    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945864    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945867    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945870    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945873    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945876    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945879    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945881    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945884    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945887    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945889    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945892    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945895    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945897    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945900    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945902    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945905    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945908    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945910    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:49.954862 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945913    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945915    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945918    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945920    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945923    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945926    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945928    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945931    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945935    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945939    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945941    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945944    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945947    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945949    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945952    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945957    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945959    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945962    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945969    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:49.955355 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945972    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945974    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945977    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945979    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945982    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945984    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945987    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945990    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945992    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945995    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.945998    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946000    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946003    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946005    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946008    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946010    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946013    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946015    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946018    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946021    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:49.955812 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946023    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946026    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946028    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946031    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946033    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946036    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946038    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946041    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946045    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946048    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946050    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946055    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946058    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946060    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946063    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946066    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946069    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946073    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946076    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946079    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:49.956420 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946083    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946085    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.946089    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.946612    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.953291    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.953307    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.953905    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.953920    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.953986    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.953991    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.953995    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.953998    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954001    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954003    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954006    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:49.956930 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954009    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954012    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954015    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954017    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954020    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954022    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954025    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954028    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954031    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954033    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954037    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954041    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954044    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954046    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954050    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954053    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954056    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954059    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954061    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:49.957333 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954064    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954068    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954071    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954084    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954088    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954091    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954093    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954096    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954099    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954102    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954105    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954108    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954111    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954114    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954117    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954119    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954122    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954125    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954128    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954131    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:49.957783 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954134    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954136    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954139    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954141    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954144    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954146    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954149    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954151    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954154    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954156    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954159    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954161    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954163    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954166    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954168    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954172    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954180    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954183    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954185    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:49.958304 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954188    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954191    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954193    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954196    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954198    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954201    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954203    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954206    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954208    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954224    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954229    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954232    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954234    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954237    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954239    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954242    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954245    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954247    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954250    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954252    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:49.958764 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954255    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954257    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954260    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954262    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954265    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954267    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954270    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954272    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954276    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954278    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954287    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954290    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954293    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954296    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.954301    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 14:18:49.959278 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954452    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954457    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954460    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954462    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954465    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954467    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954470    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954473    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954476    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954478    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954481    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954485    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954489    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954491    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954494    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954497    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954500    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954503    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954505    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 14:18:49.959646 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954508    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954511    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954514    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954516    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954519    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954522    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954524    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954527    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954530    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954539    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954542    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954544    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954547    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954549    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954552    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954554    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954557    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954559    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954562    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954564    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 14:18:49.960105 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954567    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954569    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954572    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954574    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954577    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954579    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954582    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954585    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954588    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954591    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954593    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954596    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954598    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954600    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954603    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954605    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954607    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954610    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954612    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 14:18:49.960705 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954615    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954618    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954621    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954629    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954631    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954634    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954636    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954640    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954644    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954646    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954649    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954652    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954654    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954657    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954659    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954662    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954664    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954666    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954669    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 14:18:49.961222 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954671    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954674    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954676    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954679    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954682    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954684    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954686    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954689    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954692    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954694    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954697    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954699    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954702    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954705    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954707    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954710    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954712    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954720    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954723    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954725    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 14:18:49.961676 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954728    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:49.954730    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.954735    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.954895    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.956836    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.957959    2579 server.go:1019] "Starting client certificate rotation"
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.958056    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.958090    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 14:18:49.962312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.960825    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 14:18:49.964375 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.964354    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 14:18:49.972252 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.972236    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 28 14:18:49.974353 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.974341    2579 log.go:25] "Validated CRI v1 image API"
Jul 28 14:18:49.977819 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.977801    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 14:18:49.980313 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.980294    2579 fs.go:135] Filesystem UUIDs: map[5ba35f96-d282-43cf-94a8-e0739553f624:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e4000849-e4ca-4b35-9471-4f97b0147dad:/dev/nvme0n1p4]
Jul 28 14:18:49.980377 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.980312    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 14:18:49.984441 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.984340    2579 manager.go:216] Machine: {Timestamp:2026-07-28 14:18:49.983879319 +0000 UTC m=+0.271740263 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099247 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d800697d25afa9c24f06930193eca SystemUUID:ec2d8006-97d2-5afa-9c24-f06930193eca BootID:90f9b2d4-dd96-4408-97c8-0786230ddc81 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b5:ef:92:69:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b5:ef:92:69:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:6f:95:9f:f6:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 14:18:49.984441 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.984435    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 14:18:49.984557 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.984501    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 14:18:49.986795 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.986773    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 14:18:49.986925 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.986796    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-50.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 14:18:49.988193 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.988183    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 14:18:49.988249 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.988196    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 14:18:49.988249 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.988204    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 14:18:49.988939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.988925    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 14:18:49.989762 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.989752    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 14:18:49.990837 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.990822    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 14:18:49.992393 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.992382    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 14:18:49.992513 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.992504    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 14:18:49.992545 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.992533    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 14:18:49.992572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.992546    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 14:18:49.992611 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.992571    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 14:18:49.992611 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.992586    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 28 14:18:49.992611 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.992598    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 14:18:49.993046 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.993031    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 14:18:49.993090 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.993059    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 14:18:49.994916 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.994901    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 14:18:49.996879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.996862    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 14:18:49.996934 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.996895    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 14:18:49.998818 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998806    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 14:18:49.998873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998824    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 14:18:49.998873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998830    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 14:18:49.998873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998835    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 14:18:49.998873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998843    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 14:18:49.998873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998849    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 14:18:49.998873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998868    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 14:18:49.998873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998874    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 14:18:49.999074 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998880    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 14:18:49.999074 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998885    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 14:18:49.999074 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998893    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 14:18:49.999074 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.998899    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 14:18:49.999516 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.999481    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 14:18:49.999516 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:49.999493    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 14:18:50.001637 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.001622    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-50.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 14:18:50.002088 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.002060    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 14:18:50.002088 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.002060    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-50.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 14:18:50.002589 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.002577    2579 server.go:1268] "Started kubelet"
Jul 28 14:18:50.005012 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.002706    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 14:18:50.005206 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.005185    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 14:18:50.005499 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.005423    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 14:18:50.005731 ip-10-0-141-50 systemd[1]: Started Kubernetes Kubelet.
Jul 28 14:18:50.006627 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.006590    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 14:18:50.007813 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.007792    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 14:18:50.010985 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.010958    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 14:18:50.013662 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.013648    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 14:18:50.013706 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.013663    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 14:18:50.013816 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.013741    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.014414 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014396    2579 factory.go:55] Registering systemd factory
Jul 28 14:18:50.014414 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014417    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 28 14:18:50.014553 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014426    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 14:18:50.014553 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014445    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 14:18:50.014553 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014458    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 14:18:50.014678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014578    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 14:18:50.014678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014588    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 14:18:50.014678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014635    2579 factory.go:153] Registering CRI-O factory
Jul 28 14:18:50.014678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014648    2579 factory.go:223] Registration of the crio container factory successfully
Jul 28 14:18:50.014837 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014732    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 14:18:50.014837 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014757    2579 factory.go:103] Registering Raw factory
Jul 28 14:18:50.014837 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.014773    2579 manager.go:1196] Started watching for new ooms in manager
Jul 28 14:18:50.015130 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.015119    2579 manager.go:349] Starting recovery of all containers
Jul 28 14:18:50.018887 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.018863    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dgl4j"
Jul 28 14:18:50.021910 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.021883    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-50.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 14:18:50.022003 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.021963    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 14:18:50.022872 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.022063    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-50.ec2.internal.18c679e5c1a8f474  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-50.ec2.internal,UID:ip-10-0-141-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-50.ec2.internal,},FirstTimestamp:2026-07-28 14:18:50.002527348 +0000 UTC m=+0.290388282,LastTimestamp:2026-07-28 14:18:50.002527348 +0000 UTC m=+0.290388282,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-50.ec2.internal,}"
Jul 28 14:18:50.024930 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.024912    2579 manager.go:354] Recovery completed
Jul 28 14:18:50.026283 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.026266    2579 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 14:18:50.027910 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.027896    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dgl4j"
Jul 28 14:18:50.030077 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.030052    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.032199 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.032183    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.032308 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.032299    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.032362 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.032324    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.032860 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.032845    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 14:18:50.032860 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.032860    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 14:18:50.032945 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.032879    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 14:18:50.033005 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.032981    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.033749 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.033734    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.033824 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.033758    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.033824 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.033768    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.034361 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.034289    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-50.ec2.internal.18c679e5c36db08c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-50.ec2.internal,UID:ip-10-0-141-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-50.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-50.ec2.internal,},FirstTimestamp:2026-07-28 14:18:50.032197772 +0000 UTC m=+0.320058707,LastTimestamp:2026-07-28 14:18:50.032197772 +0000 UTC m=+0.320058707,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-50.ec2.internal,}"
Jul 28 14:18:50.035415 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.035402    2579 policy_none.go:49] "None policy: Start"
Jul 28 14:18:50.035489 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.035420    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 14:18:50.035489 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.035433    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 14:18:50.037440 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.037428    2579 policy_none.go:47] "Start"
Jul 28 14:18:50.073901 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.073759    2579 manager.go:337] "Starting Device Plugin manager"
Jul 28 14:18:50.073994 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.073918    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 14:18:50.073994 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.073930    2579 server.go:87] "Starting device plugin registration server"
Jul 28 14:18:50.074143 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.074132    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 14:18:50.074194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.074146    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 14:18:50.074273 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.074257    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 14:18:50.074361 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.074349    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 14:18:50.074416 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.074362    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 14:18:50.074840 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.074820    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 14:18:50.074904 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.074872    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.174680 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.174635    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.176832 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.175614    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.176832 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.175642    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.176832 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.175652    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.176832 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.175677    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.184686 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.184662    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 14:18:50.185620 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.185605    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.185711 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.185628    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-50.ec2.internal\": node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.185918 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.185903    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 14:18:50.185956 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.185938    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 14:18:50.185989 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.185960    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 14:18:50.186034 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.185991    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 14:18:50.188306 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.188292    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:50.244444 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.244427    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.288773 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.288732    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal"]
Jul 28 14:18:50.288844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.288821    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.289794 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.289778    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.289862 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.289812    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.289862 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.289830    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.292423 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.292409    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.292561 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.292543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.292605 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.292575    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.293174 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.293157    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.293245 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.293187    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.293245 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.293201    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.293245 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.293205    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.293245 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.293238    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.293418 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.293251    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.295639 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.295622    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.295691 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.295650    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 14:18:50.296357 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.296343    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 14:18:50.296424 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.296372    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 14:18:50.296424 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.296382    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeHasSufficientPID"
Jul 28 14:18:50.316031 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.316009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d0794185faca7077a864950fb56bc94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal\" (UID: \"5d0794185faca7077a864950fb56bc94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.316130 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.316039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d0794185faca7077a864950fb56bc94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal\" (UID: \"5d0794185faca7077a864950fb56bc94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.316130 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.316064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2efc845f7ae190bc5a73ce1b56253af6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-50.ec2.internal\" (UID: \"2efc845f7ae190bc5a73ce1b56253af6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.317970 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.317951    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-50.ec2.internal\" not found" node="ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.322407 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.322392    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-50.ec2.internal\" not found" node="ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.344643 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.344620    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.416294 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.416272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d0794185faca7077a864950fb56bc94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal\" (UID: \"5d0794185faca7077a864950fb56bc94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.416416 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.416296    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d0794185faca7077a864950fb56bc94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal\" (UID: \"5d0794185faca7077a864950fb56bc94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.416416 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.416314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2efc845f7ae190bc5a73ce1b56253af6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-50.ec2.internal\" (UID: \"2efc845f7ae190bc5a73ce1b56253af6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.416416 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.416355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2efc845f7ae190bc5a73ce1b56253af6-config\") pod \"kube-apiserver-proxy-ip-10-0-141-50.ec2.internal\" (UID: \"2efc845f7ae190bc5a73ce1b56253af6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.416416 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.416395    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d0794185faca7077a864950fb56bc94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal\" (UID: \"5d0794185faca7077a864950fb56bc94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.416416 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.416395    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d0794185faca7077a864950fb56bc94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal\" (UID: \"5d0794185faca7077a864950fb56bc94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.446612 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.446584    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.549057 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.548977    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.620901 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.620866    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.626312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.626006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:50.649494 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.649463    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.751330 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.751292    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.852452 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.852380    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.923711 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.923686    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:50.953946 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:50.953916    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-50.ec2.internal\" not found"
Jul 28 14:18:50.957259 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.957242    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 14:18:50.957383 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.957355    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 14:18:50.957493 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.957388    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 14:18:50.994026 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:50.994000    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:51.014767 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.014740    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:51.014767 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.014756    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 14:18:51.033093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.033067    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 14:13:50 +0000 UTC" deadline="2028-01-30 15:20:42.887435789 +0000 UTC"
Jul 28 14:18:51.033093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.033090    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13225h1m51.85434836s"
Jul 28 14:18:51.034736 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.034723    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 14:18:51.037319 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.037280    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 14:18:51.038276 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.038262    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal"
Jul 28 14:18:51.048571 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.048554    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 14:18:51.063072 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.063051    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qml5m"
Jul 28 14:18:51.073325 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.073307    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qml5m"
Jul 28 14:18:51.383272 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:51.383242    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0794185faca7077a864950fb56bc94.slice/crio-85cb67816bb84732be659f3635451dd8c3b10d6a86d81a3339a72960249e83b1 WatchSource:0}: Error finding container 85cb67816bb84732be659f3635451dd8c3b10d6a86d81a3339a72960249e83b1: Status 404 returned error can't find the container with id 85cb67816bb84732be659f3635451dd8c3b10d6a86d81a3339a72960249e83b1
Jul 28 14:18:51.387784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.387767    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:18:51.467106 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.467080    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:51.871703 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.871673    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:51.993913 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:51.993882    2579 apiserver.go:52] "Watching apiserver"
Jul 28 14:18:52.004285 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.004265    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 14:18:52.004610 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.004591    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-9wstk","kube-system/global-pull-secret-syncer-d6pqw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5","openshift-dns/node-resolver-qc8v5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal","openshift-multus/network-metrics-daemon-xz6w2","openshift-network-operator/iptables-alerter-z9nb8","openshift-ovn-kubernetes/ovnkube-node-9qgrn","kube-system/konnectivity-agent-fz56r","kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fsz65","openshift-image-registry/node-ca-wnx4d","openshift-multus/multus-766h2","openshift-multus/multus-additional-cni-plugins-8dm86"]
Jul 28 14:18:52.008527 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.008509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.010804 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.010777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.011562 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.011541    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 14:18:52.011647 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.011623    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 14:18:52.011713 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.011694    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bpmqf\""
Jul 28 14:18:52.011979 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.011961    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 14:18:52.013118 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.013101    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.013641 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.013621    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 14:18:52.013806 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.013783    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5rndf\""
Jul 28 14:18:52.013925 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.013820    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 14:18:52.013925 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.013874    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 14:18:52.014307 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.014277    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 14:18:52.014307 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.014299    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 14:18:52.016544 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.015777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:52.016544 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.015867    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:18:52.016544 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.016158    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 14:18:52.016544 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.016255    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 14:18:52.018361 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.016808    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ck7g4\""
Jul 28 14:18:52.018361 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.016820    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 14:18:52.019527 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.019509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.021830 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.021815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:52.021896 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.021879    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:18:52.022596 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.022580    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 14:18:52.023151 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.023131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mqrkn\""
Jul 28 14:18:52.024217 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.024196    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.026684 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.026664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.027595 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cnibin\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.027683 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.027683 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.027683 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98fsk\" (UniqueName: \"kubernetes.io/projected/40008891-5ea1-483e-9743-95c20a81fec0-kube-api-access-98fsk\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.027828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:52.027828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027745    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwwnd\" (UniqueName: \"kubernetes.io/projected/3e2c8928-9386-42c8-9769-f3eaf4d71d78-kube-api-access-kwwnd\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.027828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-system-cni-dir\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.027957 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cni-binary-copy\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.027957 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.027957 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.027957 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.027957 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-socket-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.028133 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.027978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-os-release\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.028133 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.028009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-registration-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.028133 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.028044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-device-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.028133 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.028071    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-sys-fs\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.028133 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.028095    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e2c8928-9386-42c8-9769-f3eaf4d71d78-host\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.028133 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.028119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e2c8928-9386-42c8-9769-f3eaf4d71d78-serviceca\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.028360 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.028182    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6rlg\" (UniqueName: \"kubernetes.io/projected/53a7f97d-0a57-40e2-8f50-706d6cbe1235-kube-api-access-h6rlg\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.029078 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.029055    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.031543 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.031526    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.033944 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.033922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.035969 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.035952    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:18:52.036178 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.036163    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hkw49\""
Jul 28 14:18:52.036234 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.036170    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.036324 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.036306    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 14:18:52.036391 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.036310    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:18:52.037734 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.037719    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 14:18:52.037820 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.037756    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:18:52.037820 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.037796    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k86n8\""
Jul 28 14:18:52.038095 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.038081    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 14:18:52.039444 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.039428    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 14:18:52.039552 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.039536    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h5frk\""
Jul 28 14:18:52.039603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.039571    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 14:18:52.039603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.039588    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 14:18:52.040538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.040504    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 14:18:52.040636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.040537    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 14:18:52.040636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.040577    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 14:18:52.040636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.040606    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 14:18:52.040800 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.040649    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 14:18:52.040800 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.040504    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 14:18:52.040800 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.040793    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sl79r\""
Jul 28 14:18:52.041147 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.041121    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hjvwv\""
Jul 28 14:18:52.041260 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.041196    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 14:18:52.077023 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.077003    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 14:13:51 +0000 UTC" deadline="2027-12-27 03:12:34.537154881 +0000 UTC"
Jul 28 14:18:52.077023 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.077022    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12396h53m42.460135235s"
Jul 28 14:18:52.115461 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.115435    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 14:18:52.128475 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-log-socket\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.128475 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:52.128571 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-system-cni-dir\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.128571 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.128571 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-system-cni-dir\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.128708 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.128708 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-socket-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.128708 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128653    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-run\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.128708 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-kubelet\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.128708 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128686    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-os-release\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128751    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-socket-dir-parent\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-node-log\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-socket-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-os-release\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e2c8928-9386-42c8-9769-f3eaf4d71d78-host\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.128859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e2c8928-9386-42c8-9769-f3eaf4d71d78-host\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e2c8928-9386-42c8-9769-f3eaf4d71d78-serviceca\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-hostroot\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-daemon-config\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-systemd-units\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-kubelet-config\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cnibin\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98fsk\" (UniqueName: \"kubernetes.io/projected/40008891-5ea1-483e-9743-95c20a81fec0-kube-api-access-98fsk\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.128999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-modprobe-d\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129001    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cnibin\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-var-lib-kubelet\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-multus-certs\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-var-lib-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.129093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbrrw\" (UniqueName: \"kubernetes.io/projected/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-kube-api-access-mbrrw\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-registration-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-device-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-kubernetes\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn74f\" (UniqueName: \"kubernetes.io/projected/76b63489-968a-470b-9d54-6e5665beaf72-kube-api-access-fn74f\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-os-release\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129194    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kpk9\" (UniqueName: \"kubernetes.io/projected/99ae68be-8165-42d5-9579-2cc1eea8e342-kube-api-access-9kpk9\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e2c8928-9386-42c8-9769-f3eaf4d71d78-serviceca\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129290    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/857be85b-087b-4747-8d62-1a88529bba02-agent-certs\") pod \"konnectivity-agent-fz56r\" (UID: \"857be85b-087b-4747-8d62-1a88529bba02\") " pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-device-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-sys-fs\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129328    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-systemd\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-registration-dir\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129363    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76b63489-968a-470b-9d54-6e5665beaf72-tmp\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129384    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ae68be-8165-42d5-9579-2cc1eea8e342-cni-binary-copy\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129417    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-sys-fs\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.129498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-cni-netd\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-sys\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-cnibin\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-ovn\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovn-node-metrics-cert\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-iptables-alerter-script\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-host\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbbpr\" (UniqueName: \"kubernetes.io/projected/fe53c95f-e6ee-4420-bbd9-4a27000d6161-kube-api-access-lbbpr\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-cni-bin\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-env-overrides\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cni-binary-copy\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.129840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.130043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-system-cni-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovnkube-config\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-host-slash\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70597690-c107-43a7-aff6-ac3afac91223-tmp-dir\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130163    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-slash\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-systemd\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130212    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/857be85b-087b-4747-8d62-1a88529bba02-konnectivity-ca\") pod \"konnectivity-agent-fz56r\" (UID: \"857be85b-087b-4747-8d62-1a88529bba02\") " pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-lib-modules\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130286    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-cni-bin\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-cni-multus\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cni-binary-copy\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-kubelet\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bftn9\" (UniqueName: \"kubernetes.io/projected/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-kube-api-access-bftn9\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.130678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-cni-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-etc-kubernetes\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-etc-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130502    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-k8s-cni-cncf-io\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130518    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-conf-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwwnd\" (UniqueName: \"kubernetes.io/projected/3e2c8928-9386-42c8-9769-f3eaf4d71d78-kube-api-access-kwwnd\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysconfig\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysctl-d\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70597690-c107-43a7-aff6-ac3afac91223-hosts-file\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130713    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p456d\" (UniqueName: \"kubernetes.io/projected/70597690-c107-43a7-aff6-ac3afac91223-kube-api-access-p456d\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-netns\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130760    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53a7f97d-0a57-40e2-8f50-706d6cbe1235-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-run-netns\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovnkube-script-lib\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130837    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-dbus\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6rlg\" (UniqueName: \"kubernetes.io/projected/53a7f97d-0a57-40e2-8f50-706d6cbe1235-kube-api-access-h6rlg\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysctl-conf\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.131351 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.131879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.130923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76b63489-968a-470b-9d54-6e5665beaf72-etc-tuned\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.131879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.131040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/40008891-5ea1-483e-9743-95c20a81fec0-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.136175 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.136157    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:52.136246 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.136179    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:52.136246 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.136189    2579 projected.go:196] Error preparing data for projected volume kube-api-access-w2vbb for pod openshift-network-diagnostics/network-check-target-9wstk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:52.136336 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.136251    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb podName:46c7fba6-aaa2-4a69-9782-d2becb210717 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:52.6362374 +0000 UTC m=+2.924098338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w2vbb" (UniqueName: "kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb") pod "network-check-target-9wstk" (UID: "46c7fba6-aaa2-4a69-9782-d2becb210717") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:52.137364 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.137346    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 14:18:52.140579 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.140560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98fsk\" (UniqueName: \"kubernetes.io/projected/40008891-5ea1-483e-9743-95c20a81fec0-kube-api-access-98fsk\") pod \"aws-ebs-csi-driver-node-h2xn5\" (UID: \"40008891-5ea1-483e-9743-95c20a81fec0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.140579 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.140571    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwwnd\" (UniqueName: \"kubernetes.io/projected/3e2c8928-9386-42c8-9769-f3eaf4d71d78-kube-api-access-kwwnd\") pod \"node-ca-wnx4d\" (UID: \"3e2c8928-9386-42c8-9769-f3eaf4d71d78\") " pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.141513 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.141493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6rlg\" (UniqueName: \"kubernetes.io/projected/53a7f97d-0a57-40e2-8f50-706d6cbe1235-kube-api-access-h6rlg\") pod \"multus-additional-cni-plugins-8dm86\" (UID: \"53a7f97d-0a57-40e2-8f50-706d6cbe1235\") " pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.190330 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.190271    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal" event={"ID":"5d0794185faca7077a864950fb56bc94","Type":"ContainerStarted","Data":"85cb67816bb84732be659f3635451dd8c3b10d6a86d81a3339a72960249e83b1"}
Jul 28 14:18:52.231659 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysctl-conf\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.231724 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76b63489-968a-470b-9d54-6e5665beaf72-etc-tuned\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.231724 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-log-socket\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.231822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-run\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.231822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-kubelet\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.231822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-socket-dir-parent\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.231822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231783    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-log-socket\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.231822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231790    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-node-log\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysctl-conf\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-node-log\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-kubelet\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-hostroot\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-socket-dir-parent\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-daemon-config\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231864    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-run\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-systemd-units\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-hostroot\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-systemd-units\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.231964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-kubelet-config\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-modprobe-d\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232012    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-kubelet-config\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-var-lib-kubelet\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-multus-certs\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-multus-certs\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-var-lib-kubelet\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232104    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-modprobe-d\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-var-lib-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbrrw\" (UniqueName: \"kubernetes.io/projected/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-kube-api-access-mbrrw\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232175    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-kubernetes\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-var-lib-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn74f\" (UniqueName: \"kubernetes.io/projected/76b63489-968a-470b-9d54-6e5665beaf72-kube-api-access-fn74f\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-os-release\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kpk9\" (UniqueName: \"kubernetes.io/projected/99ae68be-8165-42d5-9579-2cc1eea8e342-kube-api-access-9kpk9\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232318    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-os-release\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-kubernetes\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-daemon-config\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/857be85b-087b-4747-8d62-1a88529bba02-agent-certs\") pod \"konnectivity-agent-fz56r\" (UID: \"857be85b-087b-4747-8d62-1a88529bba02\") " pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-systemd\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.232807 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76b63489-968a-470b-9d54-6e5665beaf72-tmp\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ae68be-8165-42d5-9579-2cc1eea8e342-cni-binary-copy\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-systemd\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232565    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-cni-netd\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232602    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-cni-netd\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-sys\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-cnibin\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.232693    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-ovn\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232800    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-cnibin\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovn-node-metrics-cert\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.232831    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret podName:2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:52.732808951 +0000 UTC m=+3.020669891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret") pod "global-pull-secret-syncer-d6pqw" (UID: "2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-ovn\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.233624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.232997    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-iptables-alerter-script\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-host\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbbpr\" (UniqueName: \"kubernetes.io/projected/fe53c95f-e6ee-4420-bbd9-4a27000d6161-kube-api-access-lbbpr\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-cni-bin\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233101    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-env-overrides\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233127    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-host\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-system-cni-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-system-cni-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99ae68be-8165-42d5-9579-2cc1eea8e342-cni-binary-copy\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.233409    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233427    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovnkube-config\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.233468    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs podName:fe53c95f-e6ee-4420-bbd9-4a27000d6161 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:52.73345122 +0000 UTC m=+3.021312143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs") pod "network-metrics-daemon-xz6w2" (UID: "fe53c95f-e6ee-4420-bbd9-4a27000d6161") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-cni-bin\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-env-overrides\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233747    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-sys\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.234518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-host-slash\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70597690-c107-43a7-aff6-ac3afac91223-tmp-dir\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-slash\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-systemd\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/857be85b-087b-4747-8d62-1a88529bba02-konnectivity-ca\") pod \"konnectivity-agent-fz56r\" (UID: \"857be85b-087b-4747-8d62-1a88529bba02\") " pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-iptables-alerter-script\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.233997    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-lib-modules\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234020    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-cni-bin\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234020    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovnkube-config\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-cni-multus\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-run-systemd\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-kubelet\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bftn9\" (UniqueName: \"kubernetes.io/projected/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-kube-api-access-bftn9\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234117    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-cni-multus\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-slash\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-cni-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234024    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-host-slash\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-cni-bin\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235321 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-cni-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76b63489-968a-470b-9d54-6e5665beaf72-etc-tuned\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-var-lib-kubelet\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70597690-c107-43a7-aff6-ac3afac91223-tmp-dir\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-lib-modules\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-etc-kubernetes\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-etc-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-k8s-cni-cncf-io\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-conf-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234676    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-etc-kubernetes\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-etc-openvswitch\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysconfig\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234745    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-k8s-cni-cncf-io\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234765    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysconfig\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysctl-d\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/857be85b-087b-4747-8d62-1a88529bba02-konnectivity-ca\") pod \"konnectivity-agent-fz56r\" (UID: \"857be85b-087b-4747-8d62-1a88529bba02\") " pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234829    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-multus-conf-dir\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234864    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70597690-c107-43a7-aff6-ac3afac91223-hosts-file\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.235902 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70597690-c107-43a7-aff6-ac3afac91223-hosts-file\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p456d\" (UniqueName: \"kubernetes.io/projected/70597690-c107-43a7-aff6-ac3afac91223-kube-api-access-p456d\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-netns\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-run-netns\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovnkube-script-lib\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234972    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76b63489-968a-470b-9d54-6e5665beaf72-tmp\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.234984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-dbus\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.235026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-host-run-netns\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.235041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-dbus\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.235067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99ae68be-8165-42d5-9579-2cc1eea8e342-host-run-netns\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.235142    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76b63489-968a-470b-9d54-6e5665beaf72-etc-sysctl-d\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.235381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovn-node-metrics-cert\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.235475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-ovnkube-script-lib\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.236451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.235609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/857be85b-087b-4747-8d62-1a88529bba02-agent-certs\") pod \"konnectivity-agent-fz56r\" (UID: \"857be85b-087b-4747-8d62-1a88529bba02\") " pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.246906 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.246883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn74f\" (UniqueName: \"kubernetes.io/projected/76b63489-968a-470b-9d54-6e5665beaf72-kube-api-access-fn74f\") pod \"tuned-fsz65\" (UID: \"76b63489-968a-470b-9d54-6e5665beaf72\") " pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.247238 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.247219    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kpk9\" (UniqueName: \"kubernetes.io/projected/99ae68be-8165-42d5-9579-2cc1eea8e342-kube-api-access-9kpk9\") pod \"multus-766h2\" (UID: \"99ae68be-8165-42d5-9579-2cc1eea8e342\") " pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.247621 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.247603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbbpr\" (UniqueName: \"kubernetes.io/projected/fe53c95f-e6ee-4420-bbd9-4a27000d6161-kube-api-access-lbbpr\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:52.247952 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.247934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbrrw\" (UniqueName: \"kubernetes.io/projected/a2d7a905-432c-4f7f-9a26-888ccd1ac45a-kube-api-access-mbrrw\") pod \"ovnkube-node-9qgrn\" (UID: \"a2d7a905-432c-4f7f-9a26-888ccd1ac45a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.250011 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.249989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bftn9\" (UniqueName: \"kubernetes.io/projected/74aea7d2-b2b3-4f69-bc51-4efcd22f54ea-kube-api-access-bftn9\") pod \"iptables-alerter-z9nb8\" (UID: \"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea\") " pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.250640 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.250621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p456d\" (UniqueName: \"kubernetes.io/projected/70597690-c107-43a7-aff6-ac3afac91223-kube-api-access-p456d\") pod \"node-resolver-qc8v5\" (UID: \"70597690-c107-43a7-aff6-ac3afac91223\") " pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.319726 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.319705    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wnx4d"
Jul 28 14:18:52.326413 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.326392    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8dm86"
Jul 28 14:18:52.331561 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.331540    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2c8928_9386_42c8_9769_f3eaf4d71d78.slice/crio-835ebe1ad4e392cd97435d213f04f8e4d22ac7a7eed96c26b6cae2e6e5d758da WatchSource:0}: Error finding container 835ebe1ad4e392cd97435d213f04f8e4d22ac7a7eed96c26b6cae2e6e5d758da: Status 404 returned error can't find the container with id 835ebe1ad4e392cd97435d213f04f8e4d22ac7a7eed96c26b6cae2e6e5d758da
Jul 28 14:18:52.332737 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.332716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5"
Jul 28 14:18:52.336781 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.336758    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a7f97d_0a57_40e2_8f50_706d6cbe1235.slice/crio-d91acb32d3c8ab7943613df458cd65f033f471ee334e363d234542c6909d9b30 WatchSource:0}: Error finding container d91acb32d3c8ab7943613df458cd65f033f471ee334e363d234542c6909d9b30: Status 404 returned error can't find the container with id d91acb32d3c8ab7943613df458cd65f033f471ee334e363d234542c6909d9b30
Jul 28 14:18:52.338110 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.338085    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-766h2"
Jul 28 14:18:52.343508 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.343489    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40008891_5ea1_483e_9743_95c20a81fec0.slice/crio-2868e96d1906831c26e866d41c0b8361859747ad217eccdf799d2c5155caa138 WatchSource:0}: Error finding container 2868e96d1906831c26e866d41c0b8361859747ad217eccdf799d2c5155caa138: Status 404 returned error can't find the container with id 2868e96d1906831c26e866d41c0b8361859747ad217eccdf799d2c5155caa138
Jul 28 14:18:52.344897 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.344876    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9nb8"
Jul 28 14:18:52.348317 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.348297    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ae68be_8165_42d5_9579_2cc1eea8e342.slice/crio-e6fca8fe4bd398f4f2a67501f14b84da94bdef6770c39c452d35d2f94a7d456b WatchSource:0}: Error finding container e6fca8fe4bd398f4f2a67501f14b84da94bdef6770c39c452d35d2f94a7d456b: Status 404 returned error can't find the container with id e6fca8fe4bd398f4f2a67501f14b84da94bdef6770c39c452d35d2f94a7d456b
Jul 28 14:18:52.350841 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.350769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:18:52.354109 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.354086    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74aea7d2_b2b3_4f69_bc51_4efcd22f54ea.slice/crio-c8f068a274c50177f83351b07896324c904b6b5510a87e5e8f6889aa0f82ed4e WatchSource:0}: Error finding container c8f068a274c50177f83351b07896324c904b6b5510a87e5e8f6889aa0f82ed4e: Status 404 returned error can't find the container with id c8f068a274c50177f83351b07896324c904b6b5510a87e5e8f6889aa0f82ed4e
Jul 28 14:18:52.356244 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.356203    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qc8v5"
Jul 28 14:18:52.361133 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.361111    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d7a905_432c_4f7f_9a26_888ccd1ac45a.slice/crio-33e585a8bc34e9e3944a4bd30e4942d9092c4b4b3461b934a11d51e17982871f WatchSource:0}: Error finding container 33e585a8bc34e9e3944a4bd30e4942d9092c4b4b3461b934a11d51e17982871f: Status 404 returned error can't find the container with id 33e585a8bc34e9e3944a4bd30e4942d9092c4b4b3461b934a11d51e17982871f
Jul 28 14:18:52.362981 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.362964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fsz65"
Jul 28 14:18:52.364830 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.364793    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70597690_c107_43a7_aff6_ac3afac91223.slice/crio-d831b1671566b84e65aa35d4b98e55d19402f3b05616a6576430d518e07368e5 WatchSource:0}: Error finding container d831b1671566b84e65aa35d4b98e55d19402f3b05616a6576430d518e07368e5: Status 404 returned error can't find the container with id d831b1671566b84e65aa35d4b98e55d19402f3b05616a6576430d518e07368e5
Jul 28 14:18:52.367745 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.367719    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:18:52.372080 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.372056    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b63489_968a_470b_9d54_6e5665beaf72.slice/crio-c6f4c75b4dd7e3910275c2105e512d8a937dfc2cdc287cfc3acd765a0b7513b4 WatchSource:0}: Error finding container c6f4c75b4dd7e3910275c2105e512d8a937dfc2cdc287cfc3acd765a0b7513b4: Status 404 returned error can't find the container with id c6f4c75b4dd7e3910275c2105e512d8a937dfc2cdc287cfc3acd765a0b7513b4
Jul 28 14:18:52.377390 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.377368    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857be85b_087b_4747_8d62_1a88529bba02.slice/crio-3e4ae5c791f75bcd099a9085b67d86ba059c455ffdcf89090d336e99bcc22421 WatchSource:0}: Error finding container 3e4ae5c791f75bcd099a9085b67d86ba059c455ffdcf89090d336e99bcc22421: Status 404 returned error can't find the container with id 3e4ae5c791f75bcd099a9085b67d86ba059c455ffdcf89090d336e99bcc22421
Jul 28 14:18:52.485895 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:18:52.485868    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efc845f7ae190bc5a73ce1b56253af6.slice/crio-3e74dcce176f6f1a5a96560671e3e0dac6c335977662016bdda05104c8c3ab54 WatchSource:0}: Error finding container 3e74dcce176f6f1a5a96560671e3e0dac6c335977662016bdda05104c8c3ab54: Status 404 returned error can't find the container with id 3e74dcce176f6f1a5a96560671e3e0dac6c335977662016bdda05104c8c3ab54
Jul 28 14:18:52.639429 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.639358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:52.639529 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.639489    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:52.639529 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.639509    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:52.639529 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.639523    2579 projected.go:196] Error preparing data for projected volume kube-api-access-w2vbb for pod openshift-network-diagnostics/network-check-target-9wstk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:52.639707 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.639572    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb podName:46c7fba6-aaa2-4a69-9782-d2becb210717 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:53.639558481 +0000 UTC m=+3.927419404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2vbb" (UniqueName: "kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb") pod "network-check-target-9wstk" (UID: "46c7fba6-aaa2-4a69-9782-d2becb210717") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:52.740114 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.740032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:52.740114 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:52.740101    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:52.740416 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.740216    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:52.740416 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.740218    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:52.740416 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.740293    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs podName:fe53c95f-e6ee-4420-bbd9-4a27000d6161 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:53.740274215 +0000 UTC m=+4.028135138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs") pod "network-metrics-daemon-xz6w2" (UID: "fe53c95f-e6ee-4420-bbd9-4a27000d6161") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:52.740416 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:52.740327    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret podName:2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:53.740308005 +0000 UTC m=+4.028168934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret") pod "global-pull-secret-syncer-d6pqw" (UID: "2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:53.077778 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.077615    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 14:13:51 +0000 UTC" deadline="2028-02-22 08:51:41.034050632 +0000 UTC"
Jul 28 14:18:53.077778 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.077679    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13770h32m47.956375923s"
Jul 28 14:18:53.186571 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.186537    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:53.186771 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.186677    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:18:53.213499 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.213401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5" event={"ID":"40008891-5ea1-483e-9743-95c20a81fec0","Type":"ContainerStarted","Data":"2868e96d1906831c26e866d41c0b8361859747ad217eccdf799d2c5155caa138"}
Jul 28 14:18:53.236519 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.236482    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wnx4d" event={"ID":"3e2c8928-9386-42c8-9769-f3eaf4d71d78","Type":"ContainerStarted","Data":"835ebe1ad4e392cd97435d213f04f8e4d22ac7a7eed96c26b6cae2e6e5d758da"}
Jul 28 14:18:53.257464 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.257163    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qc8v5" event={"ID":"70597690-c107-43a7-aff6-ac3afac91223","Type":"ContainerStarted","Data":"d831b1671566b84e65aa35d4b98e55d19402f3b05616a6576430d518e07368e5"}
Jul 28 14:18:53.268235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.268064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"33e585a8bc34e9e3944a4bd30e4942d9092c4b4b3461b934a11d51e17982871f"}
Jul 28 14:18:53.272621 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.272594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9nb8" event={"ID":"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea","Type":"ContainerStarted","Data":"c8f068a274c50177f83351b07896324c904b6b5510a87e5e8f6889aa0f82ed4e"}
Jul 28 14:18:53.279876 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.279848    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerStarted","Data":"d91acb32d3c8ab7943613df458cd65f033f471ee334e363d234542c6909d9b30"}
Jul 28 14:18:53.290145 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.290119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal" event={"ID":"2efc845f7ae190bc5a73ce1b56253af6","Type":"ContainerStarted","Data":"3e74dcce176f6f1a5a96560671e3e0dac6c335977662016bdda05104c8c3ab54"}
Jul 28 14:18:53.311788 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.311520    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fz56r" event={"ID":"857be85b-087b-4747-8d62-1a88529bba02","Type":"ContainerStarted","Data":"3e4ae5c791f75bcd099a9085b67d86ba059c455ffdcf89090d336e99bcc22421"}
Jul 28 14:18:53.339339 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.339237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fsz65" event={"ID":"76b63489-968a-470b-9d54-6e5665beaf72","Type":"ContainerStarted","Data":"c6f4c75b4dd7e3910275c2105e512d8a937dfc2cdc287cfc3acd765a0b7513b4"}
Jul 28 14:18:53.359271 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.359237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-766h2" event={"ID":"99ae68be-8165-42d5-9579-2cc1eea8e342","Type":"ContainerStarted","Data":"e6fca8fe4bd398f4f2a67501f14b84da94bdef6770c39c452d35d2f94a7d456b"}
Jul 28 14:18:53.656777 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.656685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:53.656962 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.656891    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:53.656962 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.656916    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:53.656962 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.656929    2579 projected.go:196] Error preparing data for projected volume kube-api-access-w2vbb for pod openshift-network-diagnostics/network-check-target-9wstk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:53.657132 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.656994    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb podName:46c7fba6-aaa2-4a69-9782-d2becb210717 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:55.656974867 +0000 UTC m=+5.944835792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2vbb" (UniqueName: "kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb") pod "network-check-target-9wstk" (UID: "46c7fba6-aaa2-4a69-9782-d2becb210717") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:53.758070 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.758032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:53.758264 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:53.758092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:53.758405 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.758273    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:53.758405 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.758339    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs podName:fe53c95f-e6ee-4420-bbd9-4a27000d6161 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:55.758319474 +0000 UTC m=+6.046180399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs") pod "network-metrics-daemon-xz6w2" (UID: "fe53c95f-e6ee-4420-bbd9-4a27000d6161") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:53.758765 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.758746    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:53.758834 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:53.758805    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret podName:2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:55.758788401 +0000 UTC m=+6.046649326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret") pod "global-pull-secret-syncer-d6pqw" (UID: "2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:54.081216 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:54.081179    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 14:18:54.189828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:54.189793    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:54.190021 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:54.189940    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:18:54.190456 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:54.190390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:54.190555 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:54.190499    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:18:55.186364 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:55.186255    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:55.186950 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.186404    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:18:55.676962 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:55.676213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:55.676962 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.676468    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:55.676962 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.676492    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:55.676962 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.676505    2579 projected.go:196] Error preparing data for projected volume kube-api-access-w2vbb for pod openshift-network-diagnostics/network-check-target-9wstk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:55.676962 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.676564    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb podName:46c7fba6-aaa2-4a69-9782-d2becb210717 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:59.676545094 +0000 UTC m=+9.964406019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2vbb" (UniqueName: "kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb") pod "network-check-target-9wstk" (UID: "46c7fba6-aaa2-4a69-9782-d2becb210717") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:55.781107 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:55.777585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:55.781107 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:55.777636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:55.781107 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.777809    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:55.781107 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.777876    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs podName:fe53c95f-e6ee-4420-bbd9-4a27000d6161 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:59.777856063 +0000 UTC m=+10.065716990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs") pod "network-metrics-daemon-xz6w2" (UID: "fe53c95f-e6ee-4420-bbd9-4a27000d6161") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:55.781107 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.778305    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:55.781107 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:55.778358    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret podName:2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6 nodeName:}" failed. No retries permitted until 2026-07-28 14:18:59.77834238 +0000 UTC m=+10.066203307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret") pod "global-pull-secret-syncer-d6pqw" (UID: "2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:56.186757 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:56.186519    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:56.186757 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:56.186690    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:56.187326 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:56.186780    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:18:56.187326 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:56.186786    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:18:57.187366 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:57.186818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:57.187366 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:57.186961    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:18:58.186861 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:58.186827    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:58.187024 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:58.186944    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:18:58.187392 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:58.187256    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:58.187392 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:58.187357    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:18:59.186998 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:59.186968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:59.187149 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.187102    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:18:59.711650 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:59.711612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:18:59.712056 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.711812    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:18:59.712056 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.711838    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:18:59.712056 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.711851    2579 projected.go:196] Error preparing data for projected volume kube-api-access-w2vbb for pod openshift-network-diagnostics/network-check-target-9wstk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:59.712056 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.711916    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb podName:46c7fba6-aaa2-4a69-9782-d2becb210717 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:07.711897784 +0000 UTC m=+17.999758720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2vbb" (UniqueName: "kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb") pod "network-check-target-9wstk" (UID: "46c7fba6-aaa2-4a69-9782-d2becb210717") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:18:59.812859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:59.812825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:18:59.813037 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:18:59.812936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:18:59.813037 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.813011    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:59.813149 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.813069    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret podName:2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:07.813051179 +0000 UTC m=+18.100912103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret") pod "global-pull-secret-syncer-d6pqw" (UID: "2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:18:59.813149 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.813011    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:18:59.813149 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:18:59.813125    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs podName:fe53c95f-e6ee-4420-bbd9-4a27000d6161 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:07.813112535 +0000 UTC m=+18.100973459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs") pod "network-metrics-daemon-xz6w2" (UID: "fe53c95f-e6ee-4420-bbd9-4a27000d6161") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:00.190560 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:00.190480    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:00.190560 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:00.190521    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:00.190767 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:00.190629    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:00.190767 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:00.190753    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:01.187267 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:01.187232    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:01.187708 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:01.187391    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:02.186819 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:02.186790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:02.186960 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:02.186898    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:02.186960 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:02.186950    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:02.187055 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:02.187039    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:03.186471 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:03.186444    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:03.186835 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:03.186538    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:04.187007 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:04.186973    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:04.187498 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:04.186980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:04.187498 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:04.187113    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:04.187498 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:04.187200    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:05.186701 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:05.186624    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:05.186847 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:05.186755    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:06.187147 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:06.187118    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:06.187599 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:06.187277    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:06.187599 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:06.187336    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:06.187599 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:06.187447    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:07.186960 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:07.186925    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:07.187168 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.187035    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:07.769601 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:07.769560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:07.769977 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.769756    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:19:07.769977 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.769785    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:19:07.769977 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.769798    2579 projected.go:196] Error preparing data for projected volume kube-api-access-w2vbb for pod openshift-network-diagnostics/network-check-target-9wstk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:07.769977 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.769860    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb podName:46c7fba6-aaa2-4a69-9782-d2becb210717 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:23.769842457 +0000 UTC m=+34.057703391 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2vbb" (UniqueName: "kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb") pod "network-check-target-9wstk" (UID: "46c7fba6-aaa2-4a69-9782-d2becb210717") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:07.870496 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:07.870455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:07.870663 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:07.870519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:07.870663 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.870622    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:07.870663 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.870661    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:07.870775 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.870687    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret podName:2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:23.870669698 +0000 UTC m=+34.158530633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret") pod "global-pull-secret-syncer-d6pqw" (UID: "2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:07.870775 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:07.870704    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs podName:fe53c95f-e6ee-4420-bbd9-4a27000d6161 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:23.87069584 +0000 UTC m=+34.158556761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs") pod "network-metrics-daemon-xz6w2" (UID: "fe53c95f-e6ee-4420-bbd9-4a27000d6161") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:08.186384 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:08.186284    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:08.186384 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:08.186311    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:08.186603 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:08.186412    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:08.186603 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:08.186503    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:09.187084 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:09.187053    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:09.187496 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:09.187155    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:10.187648 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:10.187609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:10.188058 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:10.187726    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:10.188058 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:10.187784    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:10.188058 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:10.187904    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:11.186561 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:11.186528    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:11.186725 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:11.186660    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:12.187015 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:12.186978    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:12.187628 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:12.187101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:12.187628 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:12.187184    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:12.187628 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:12.187323    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:13.187389 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.187089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:13.188060 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:13.187487    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:13.410488 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.410455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal" event={"ID":"2efc845f7ae190bc5a73ce1b56253af6","Type":"ContainerStarted","Data":"3c4ac2dbef10b74d062e9a88889165ba55e5891af6979418f2ea0100f710dfdf"}
Jul 28 14:19:13.411775 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.411748    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fz56r" event={"ID":"857be85b-087b-4747-8d62-1a88529bba02","Type":"ContainerStarted","Data":"b416f55e14701e0b6a68c2af3d85664a85885105eca28c11ccad642e9fefcce5"}
Jul 28 14:19:13.413099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.413078    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fsz65" event={"ID":"76b63489-968a-470b-9d54-6e5665beaf72","Type":"ContainerStarted","Data":"9d6a11c4769fc8b9eb73eeb20ce83fc742c0ea5d6535412741d7797cd6a08c99"}
Jul 28 14:19:13.414605 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.414323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-766h2" event={"ID":"99ae68be-8165-42d5-9579-2cc1eea8e342","Type":"ContainerStarted","Data":"876904f1a0644e9d42e039aeb1a3fcf093f9e2a9af871c46579bac241f0bcb5d"}
Jul 28 14:19:13.418629 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.418608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5" event={"ID":"40008891-5ea1-483e-9743-95c20a81fec0","Type":"ContainerStarted","Data":"c761438fae238ee0fdd8462b4dae038c5071ff99792facb06216ca21668b5f53"}
Jul 28 14:19:13.419759 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.419737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wnx4d" event={"ID":"3e2c8928-9386-42c8-9769-f3eaf4d71d78","Type":"ContainerStarted","Data":"a2191cb7d9b710325f13279743db44ab9d66ffe36107832bea253fdae7313cc9"}
Jul 28 14:19:13.421040 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.421022    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qc8v5" event={"ID":"70597690-c107-43a7-aff6-ac3afac91223","Type":"ContainerStarted","Data":"a4af9ca99f07890db11d932f2cb5f6ba44690ed8bf40ad0dde750e059f6e7fb0"}
Jul 28 14:19:13.423673 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.423653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"bac81a0183574cb301b4ae671a2664d2916d07b4d53169a7e06287844a7c6fff"}
Jul 28 14:19:13.423767 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.423718    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"a26b8291c7fbd9a74c05bd671d415119d75c2d2e608ca36e187e06890518ee66"}
Jul 28 14:19:13.423828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.423768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"5380b21d43de1b904a535a899148853bbe4dc8f2c072a097921bf7fd48549773"}
Jul 28 14:19:13.423828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.423817    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"2c01f71755b33a7ad0880cb111ca7aad7788dfa18cf4ee7e17e8b907c1e0135c"}
Jul 28 14:19:13.423943 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.423864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"646e66c6cefad159a8aed9729e437183202fe792e10c82bf77fb7d18e2f4c297"}
Jul 28 14:19:13.423943 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.423913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"b215a643d81b9133f193800e7339b9a38cdd4652dac0ec33d86f8fef2f41386e"}
Jul 28 14:19:13.424914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.424896    2579 generic.go:356] "Generic (PLEG): container finished" podID="53a7f97d-0a57-40e2-8f50-706d6cbe1235" containerID="0aa2002a2e8b5476945c693c502dbbd0202a0d23c746b40c7ae1c4109ad3c048" exitCode=0
Jul 28 14:19:13.424995 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.424951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerDied","Data":"0aa2002a2e8b5476945c693c502dbbd0202a0d23c746b40c7ae1c4109ad3c048"}
Jul 28 14:19:13.426401 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.426377    2579 generic.go:356] "Generic (PLEG): container finished" podID="5d0794185faca7077a864950fb56bc94" containerID="c4c3f593eab7b4e022aad5ec619e0f07e3e720a7bcea0bfa253af7bcee42e6cd" exitCode=0
Jul 28 14:19:13.426505 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.426411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal" event={"ID":"5d0794185faca7077a864950fb56bc94","Type":"ContainerDied","Data":"c4c3f593eab7b4e022aad5ec619e0f07e3e720a7bcea0bfa253af7bcee42e6cd"}
Jul 28 14:19:13.427919 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.427875    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-50.ec2.internal" podStartSLOduration=22.427859906 podStartE2EDuration="22.427859906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.42772519 +0000 UTC m=+23.715586156" watchObservedRunningTime="2026-07-28 14:19:13.427859906 +0000 UTC m=+23.715720854"
Jul 28 14:19:13.462575 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.462419    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wnx4d" podStartSLOduration=3.417893759 podStartE2EDuration="23.462406173s" totalImagesPullingTime="20.044512414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.46197444 +0000 UTC m=+23.749835401" watchObservedRunningTime="2026-07-28 14:19:13.462406173 +0000 UTC m=+23.750267115"
Jul 28 14:19:13.513443 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.513403    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-766h2" podStartSLOduration=3.434087433 podStartE2EDuration="23.513384857s" totalImagesPullingTime="20.079297424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.512868418 +0000 UTC m=+23.800729381" watchObservedRunningTime="2026-07-28 14:19:13.513384857 +0000 UTC m=+23.801245799"
Jul 28 14:19:13.529678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.529630    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qc8v5" podStartSLOduration=2.504678657 podStartE2EDuration="22.5296123s" totalImagesPullingTime="20.024933643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.529557037 +0000 UTC m=+23.817417981" watchObservedRunningTime="2026-07-28 14:19:13.5296123 +0000 UTC m=+23.817473245"
Jul 28 14:19:13.551607 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.551567    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fz56r" podStartSLOduration=2.552052051 podStartE2EDuration="22.551549553s" totalImagesPullingTime="19.999497502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.550868543 +0000 UTC m=+23.838729510" watchObservedRunningTime="2026-07-28 14:19:13.551549553 +0000 UTC m=+23.839410497"
Jul 28 14:19:13.584566 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:13.584504    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fsz65" podStartSLOduration=2.574251338 podStartE2EDuration="22.584482113s" totalImagesPullingTime="20.010230775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:13.583591076 +0000 UTC m=+23.871452034" watchObservedRunningTime="2026-07-28 14:19:13.584482113 +0000 UTC m=+23.872343063"
Jul 28 14:19:14.186887 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.186857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:14.187081 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:14.186965    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:14.187081 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.187022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:14.187151 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:14.187127    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:14.344705 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.344684    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 14:19:14.430082 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.430038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5" event={"ID":"40008891-5ea1-483e-9743-95c20a81fec0","Type":"ContainerStarted","Data":"2b826bdbf95f305c3ab2e1e63e0bde9903702167b2e15081ba4d15fa86eb8c4f"}
Jul 28 14:19:14.431527 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.431501    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9nb8" event={"ID":"74aea7d2-b2b3-4f69-bc51-4efcd22f54ea","Type":"ContainerStarted","Data":"b0755a6c26fe332f3040be5e4b9ac2d8ad13f95012e4aa3f187aa00ec74958c0"}
Jul 28 14:19:14.433111 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.433084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal" event={"ID":"5d0794185faca7077a864950fb56bc94","Type":"ContainerStarted","Data":"5a243fcaee0b8fc215860e48a574d0d95e0c071ab118f8365fac6ec8955cb6ae"}
Jul 28 14:19:14.449653 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.449586    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z9nb8" podStartSLOduration=4.432774326 podStartE2EDuration="24.449572694s" totalImagesPullingTime="20.016798368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:14.449245132 +0000 UTC m=+24.737106094" watchObservedRunningTime="2026-07-28 14:19:14.449572694 +0000 UTC m=+24.737433637"
Jul 28 14:19:14.471159 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:14.471116    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-50.ec2.internal" podStartSLOduration=23.471104253 podStartE2EDuration="23.471104253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:14.470707246 +0000 UTC m=+24.758568189" watchObservedRunningTime="2026-07-28 14:19:14.471104253 +0000 UTC m=+24.758965196"
Jul 28 14:19:15.092045 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:15.091936    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T14:19:14.3447016Z","UUID":"7bf948f3-82f3-4869-a776-03bbbe246747","Handler":null,"Name":"","Endpoint":""}
Jul 28 14:19:15.094927 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:15.094902    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 14:19:15.095062 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:15.094942    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 14:19:15.187176 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:15.187151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:15.187342 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:15.187287    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:15.440261 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:15.440160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"a41ceaebb6bc3df01a021c4fc13a0fd7f958c8d419eb5b30f81bbd9d23eba9d7"}
Jul 28 14:19:16.186835 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:16.186798    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:16.187017 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:16.186918    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:16.187017 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:16.186958    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:16.187132 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:16.187069    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:16.444366 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:16.444288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5" event={"ID":"40008891-5ea1-483e-9743-95c20a81fec0","Type":"ContainerStarted","Data":"918620582cf6d93d3f839b76a5a961771d4ca5f2711915db2dc24a8ff6a493e3"}
Jul 28 14:19:16.465841 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:16.465797    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2xn5" podStartSLOduration=3.735140314 podStartE2EDuration="26.465778134s" totalImagesPullingTime="22.73063782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:16.465364776 +0000 UTC m=+26.753225721" watchObservedRunningTime="2026-07-28 14:19:16.465778134 +0000 UTC m=+26.753639067"
Jul 28 14:19:17.186483 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:17.186452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:17.186652 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:17.186585    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:18.187114 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.186926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:18.187822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.186929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:18.187822 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:18.187200    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:18.187822 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:18.187330    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:18.362844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.362746    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:19:18.363462 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.363438    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:19:18.452887 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.452833    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" event={"ID":"a2d7a905-432c-4f7f-9a26-888ccd1ac45a","Type":"ContainerStarted","Data":"502d8c8418c07be5101fe9792832dbb94abcf749bfe4fac104d7d5a403c09dd0"}
Jul 28 14:19:18.453285 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.453243    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:19:18.453752 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.453733    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fz56r"
Jul 28 14:19:18.484785 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:18.484744    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn" podStartSLOduration=8.397407035 podStartE2EDuration="28.484727841s" totalImagesPullingTime="20.087320806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:18.48287715 +0000 UTC m=+28.770738111" watchObservedRunningTime="2026-07-28 14:19:18.484727841 +0000 UTC m=+28.772588797"
Jul 28 14:19:19.186581 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.186553    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:19.186731 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:19.186644    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:19.458018 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.457726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerStarted","Data":"fa2db2b4e60d0fc1713a276a75e9605eea7b1f9259266a3f96ede128e74664e2"}
Jul 28 14:19:19.458556 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.458476    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:19:19.458556 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.458549    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:19:19.458665 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.458601    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:19:19.473787 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.473767    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:19:19.473897 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.473863    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:19:19.601425 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.601388    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9wstk"]
Jul 28 14:19:19.601573 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.601511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:19.601631 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:19.601610    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:19.603919 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.603894    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d6pqw"]
Jul 28 14:19:19.604034 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.603998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:19.604109 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:19.604092    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:19.615646 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.615623    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xz6w2"]
Jul 28 14:19:19.615760 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:19.615730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:19.615852 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:19.615831    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:20.461623 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:20.461592    2579 generic.go:356] "Generic (PLEG): container finished" podID="53a7f97d-0a57-40e2-8f50-706d6cbe1235" containerID="fa2db2b4e60d0fc1713a276a75e9605eea7b1f9259266a3f96ede128e74664e2" exitCode=0
Jul 28 14:19:20.461986 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:20.461678    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerDied","Data":"fa2db2b4e60d0fc1713a276a75e9605eea7b1f9259266a3f96ede128e74664e2"}
Jul 28 14:19:21.187044 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:21.187010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:21.187044 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:21.187039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:21.187305 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:21.187010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:21.187305 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:21.187101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:21.189562 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:21.187588    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:21.189562 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:21.187644    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:22.468910 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:22.468833    2579 generic.go:356] "Generic (PLEG): container finished" podID="53a7f97d-0a57-40e2-8f50-706d6cbe1235" containerID="9e003da8659d4923da729af97621b1772aebe6b1c104341e8ca4e7265c644825" exitCode=0
Jul 28 14:19:22.468910 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:22.468880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerDied","Data":"9e003da8659d4923da729af97621b1772aebe6b1c104341e8ca4e7265c644825"}
Jul 28 14:19:23.186783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.186755    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:23.186894 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.186755    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:23.186943 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.186755    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:23.186943 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.186933    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:23.187016 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.186847    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9wstk" podUID="46c7fba6-aaa2-4a69-9782-d2becb210717"
Jul 28 14:19:23.187068 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.187003    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6pqw" podUID="2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6"
Jul 28 14:19:23.473531 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.473497    2579 generic.go:356] "Generic (PLEG): container finished" podID="53a7f97d-0a57-40e2-8f50-706d6cbe1235" containerID="ebf865e3a1c9c090b7b0f58bcf6122c9a7acc6373173b54bcfcf5d55876f1b6a" exitCode=0
Jul 28 14:19:23.473918 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.473536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerDied","Data":"ebf865e3a1c9c090b7b0f58bcf6122c9a7acc6373173b54bcfcf5d55876f1b6a"}
Jul 28 14:19:23.788495 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.788467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:23.788658 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.788582    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 14:19:23.788658 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.788595    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 14:19:23.788658 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.788603    2579 projected.go:196] Error preparing data for projected volume kube-api-access-w2vbb for pod openshift-network-diagnostics/network-check-target-9wstk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:23.788658 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.788644    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb podName:46c7fba6-aaa2-4a69-9782-d2becb210717 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:55.788632713 +0000 UTC m=+66.076493634 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2vbb" (UniqueName: "kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb") pod "network-check-target-9wstk" (UID: "46c7fba6-aaa2-4a69-9782-d2becb210717") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 14:19:23.888844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.888810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:23.889017 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:23.888860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:23.889017 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.888967    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:23.889017 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.888989    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:23.889172 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.889029    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs podName:fe53c95f-e6ee-4420-bbd9-4a27000d6161 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:55.889010495 +0000 UTC m=+66.176871416 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs") pod "network-metrics-daemon-xz6w2" (UID: "fe53c95f-e6ee-4420-bbd9-4a27000d6161") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 14:19:23.889172 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:23.889046    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret podName:2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6 nodeName:}" failed. No retries permitted until 2026-07-28 14:19:55.889037348 +0000 UTC m=+66.176898269 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret") pod "global-pull-secret-syncer-d6pqw" (UID: "2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6") : object "kube-system"/"original-pull-secret" not registered
Jul 28 14:19:24.190202 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:24.190172    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:24.190368 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:24.190334    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xz6w2" podUID="fe53c95f-e6ee-4420-bbd9-4a27000d6161"
Jul 28 14:19:25.064860 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.064664    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-50.ec2.internal" event="NodeReady"
Jul 28 14:19:25.065294 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.064977    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 14:19:25.117143 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.117115    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-br86x"]
Jul 28 14:19:25.120126 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.120103    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hxz4r"]
Jul 28 14:19:25.120297 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.120277    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.123376 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.123349    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 14:19:25.123586 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.123567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zfw4b\""
Jul 28 14:19:25.123879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.123696    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 14:19:25.123879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.123825    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 14:19:25.124060 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.123927    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.126808 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.126755    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 14:19:25.126976 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.126756    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 14:19:25.126976 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.126765    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6f28j\""
Jul 28 14:19:25.129340 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.129310    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-br86x"]
Jul 28 14:19:25.138157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.138138    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hxz4r"]
Jul 28 14:19:25.186555 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.186532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:25.186728 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.186710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:25.189973 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.189949    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v4vsp\""
Jul 28 14:19:25.190068 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.189957    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 14:19:25.190141 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.190125    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 14:19:25.190255 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.190239    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 14:19:25.300008 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.299977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-564t7\" (UniqueName: \"kubernetes.io/projected/9bcedd50-8330-4d98-98c7-1df1500b80bb-kube-api-access-564t7\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.300159 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.300016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.300159 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.300035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f44c\" (UniqueName: \"kubernetes.io/projected/d77ee87e-9bae-4382-bee7-14f28427ff6b-kube-api-access-7f44c\") pod \"ingress-canary-br86x\" (UID: \"d77ee87e-9bae-4382-bee7-14f28427ff6b\") " pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.300159 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.300050    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcedd50-8330-4d98-98c7-1df1500b80bb-config-volume\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.300159 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.300122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d77ee87e-9bae-4382-bee7-14f28427ff6b-cert\") pod \"ingress-canary-br86x\" (UID: \"d77ee87e-9bae-4382-bee7-14f28427ff6b\") " pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.300159 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.300156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcedd50-8330-4d98-98c7-1df1500b80bb-tmp-dir\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.401341 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.401306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-564t7\" (UniqueName: \"kubernetes.io/projected/9bcedd50-8330-4d98-98c7-1df1500b80bb-kube-api-access-564t7\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.401516 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.401357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.401516 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:25.401457    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:25.401516 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.401477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f44c\" (UniqueName: \"kubernetes.io/projected/d77ee87e-9bae-4382-bee7-14f28427ff6b-kube-api-access-7f44c\") pod \"ingress-canary-br86x\" (UID: \"d77ee87e-9bae-4382-bee7-14f28427ff6b\") " pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.401516 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:25.401515    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls podName:9bcedd50-8330-4d98-98c7-1df1500b80bb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:25.901495709 +0000 UTC m=+36.189356633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls") pod "dns-default-hxz4r" (UID: "9bcedd50-8330-4d98-98c7-1df1500b80bb") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:25.401703 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.401543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcedd50-8330-4d98-98c7-1df1500b80bb-config-volume\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.401703 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.401570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d77ee87e-9bae-4382-bee7-14f28427ff6b-cert\") pod \"ingress-canary-br86x\" (UID: \"d77ee87e-9bae-4382-bee7-14f28427ff6b\") " pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.401703 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.401593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcedd50-8330-4d98-98c7-1df1500b80bb-tmp-dir\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.402229 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.402188    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcedd50-8330-4d98-98c7-1df1500b80bb-tmp-dir\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.402471 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.402444    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcedd50-8330-4d98-98c7-1df1500b80bb-config-volume\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.406482 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.406461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d77ee87e-9bae-4382-bee7-14f28427ff6b-cert\") pod \"ingress-canary-br86x\" (UID: \"d77ee87e-9bae-4382-bee7-14f28427ff6b\") " pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.411056 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.411033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-564t7\" (UniqueName: \"kubernetes.io/projected/9bcedd50-8330-4d98-98c7-1df1500b80bb-kube-api-access-564t7\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.411206 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.411170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f44c\" (UniqueName: \"kubernetes.io/projected/d77ee87e-9bae-4382-bee7-14f28427ff6b-kube-api-access-7f44c\") pod \"ingress-canary-br86x\" (UID: \"d77ee87e-9bae-4382-bee7-14f28427ff6b\") " pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.432473 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.432454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-br86x"
Jul 28 14:19:25.581395 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.581367    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-br86x"]
Jul 28 14:19:25.585904 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:25.585877    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd77ee87e_9bae_4382_bee7_14f28427ff6b.slice/crio-d3d8b1913f2dd97d5d716e2a6f4eabaff67c1bd53b4a16cce2718d34fd3e9f3a WatchSource:0}: Error finding container d3d8b1913f2dd97d5d716e2a6f4eabaff67c1bd53b4a16cce2718d34fd3e9f3a: Status 404 returned error can't find the container with id d3d8b1913f2dd97d5d716e2a6f4eabaff67c1bd53b4a16cce2718d34fd3e9f3a
Jul 28 14:19:25.905085 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:25.905000    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:25.905365 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:25.905166    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:25.905365 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:25.905237    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls podName:9bcedd50-8330-4d98-98c7-1df1500b80bb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:26.905208109 +0000 UTC m=+37.193069036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls") pod "dns-default-hxz4r" (UID: "9bcedd50-8330-4d98-98c7-1df1500b80bb") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:26.186342 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.186314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:26.190245 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.189557    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gd6j\""
Jul 28 14:19:26.190245 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.189855    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 14:19:26.483712 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.483624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-br86x" event={"ID":"d77ee87e-9bae-4382-bee7-14f28427ff6b","Type":"ContainerStarted","Data":"d3d8b1913f2dd97d5d716e2a6f4eabaff67c1bd53b4a16cce2718d34fd3e9f3a"}
Jul 28 14:19:26.669947 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.669912    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8sngc"]
Jul 28 14:19:26.673824 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.673797    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.687647 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.687626    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 14:19:26.687775 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.687745    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 14:19:26.689121 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.689096    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 14:19:26.689226 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.689164    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kr6cm\""
Jul 28 14:19:26.689298 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.689165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 14:19:26.706101 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.706079    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8sngc"]
Jul 28 14:19:26.811880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.811794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.811880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.811862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/307891dc-8150-4d47-8fa3-849b4b25c3dc-crio-socket\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.812108 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.811970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.812108 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.812026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5h7r\" (UniqueName: \"kubernetes.io/projected/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-api-access-d5h7r\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.812108 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.812083    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/307891dc-8150-4d47-8fa3-849b4b25c3dc-data-volume\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.912635 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.912606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/307891dc-8150-4d47-8fa3-849b4b25c3dc-crio-socket\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.912818 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.912649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:26.912818 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.912677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.912818 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.912710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5h7r\" (UniqueName: \"kubernetes.io/projected/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-api-access-d5h7r\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.912818 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.912742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/307891dc-8150-4d47-8fa3-849b4b25c3dc-data-volume\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.912818 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.912781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.913067 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:26.912830    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:26.913067 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.912859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/307891dc-8150-4d47-8fa3-849b4b25c3dc-crio-socket\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.913067 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:26.912876    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:26.913067 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:26.912912    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:26.913067 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:26.912939    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:27.412921645 +0000 UTC m=+37.700782567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:26.913067 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:26.912957    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls podName:9bcedd50-8330-4d98-98c7-1df1500b80bb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:28.912948013 +0000 UTC m=+39.200808937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls") pod "dns-default-hxz4r" (UID: "9bcedd50-8330-4d98-98c7-1df1500b80bb") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:26.913067 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:26.912980    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:27.412963015 +0000 UTC m=+37.700823936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:26.913450 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.913137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/307891dc-8150-4d47-8fa3-849b4b25c3dc-data-volume\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:26.924477 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:26.924451    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5h7r\" (UniqueName: \"kubernetes.io/projected/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-api-access-d5h7r\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:27.416984 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:27.416946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:27.417360 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:27.417015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:27.417360 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:27.417071    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:27.417360 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:27.417107    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:27.417360 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:27.417133    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:28.417118196 +0000 UTC m=+38.704979120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:27.417360 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:27.417148    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:28.417142234 +0000 UTC m=+38.705003155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:28.424725 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:28.424474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:28.425169 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:28.424598    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:28.425169 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:28.424751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:28.425169 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:28.424835    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:30.42480315 +0000 UTC m=+40.712664085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:28.425169 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:28.424851    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:28.425169 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:28.424907    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:30.42489014 +0000 UTC m=+40.712751064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:28.492047 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:28.492020    2579 generic.go:356] "Generic (PLEG): container finished" podID="53a7f97d-0a57-40e2-8f50-706d6cbe1235" containerID="7ce6c7de32542b2bd5e0e5b43b6cd1c2fc62b208c67680eb5ab0739454af6725" exitCode=0
Jul 28 14:19:28.492047 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:28.492057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerDied","Data":"7ce6c7de32542b2bd5e0e5b43b6cd1c2fc62b208c67680eb5ab0739454af6725"}
Jul 28 14:19:28.858366 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:28.858330    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qc8v5_70597690-c107-43a7-aff6-ac3afac91223/dns-node-resolver/0.log"
Jul 28 14:19:28.928926 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:28.928888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:28.929096 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:28.929058    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:28.929175 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:28.929160    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls podName:9bcedd50-8330-4d98-98c7-1df1500b80bb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:32.92913638 +0000 UTC m=+43.216997303 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls") pod "dns-default-hxz4r" (UID: "9bcedd50-8330-4d98-98c7-1df1500b80bb") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:29.497680 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:29.497649    2579 generic.go:356] "Generic (PLEG): container finished" podID="53a7f97d-0a57-40e2-8f50-706d6cbe1235" containerID="81d226044842f406564160685f879d0a63982e1f9fb79fb44ae132a578552d40" exitCode=0
Jul 28 14:19:29.498021 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:29.497714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerDied","Data":"81d226044842f406564160685f879d0a63982e1f9fb79fb44ae132a578552d40"}
Jul 28 14:19:29.657577 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:29.657560    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wnx4d_3e2c8928-9386-42c8-9769-f3eaf4d71d78/node-ca/0.log"
Jul 28 14:19:30.442437 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:30.442399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:30.442631 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:30.442484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:30.442631 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:30.442517    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:30.442631 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:30.442580    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:34.442565789 +0000 UTC m=+44.730426710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:30.442631 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:30.442607    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:30.442808 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:30.442656    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:34.442640846 +0000 UTC m=+44.730501769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:30.501804 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:30.501770    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-br86x" event={"ID":"d77ee87e-9bae-4382-bee7-14f28427ff6b","Type":"ContainerStarted","Data":"1eff14db609a56c71e562c3930ee86c3b0a4990e0302fd22bffea74e8501f23e"}
Jul 28 14:19:30.505307 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:30.505272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dm86" event={"ID":"53a7f97d-0a57-40e2-8f50-706d6cbe1235","Type":"ContainerStarted","Data":"852b0ff4ec9c6eb499b67dfb74a1e537cd957315ca3b4a9f0c34a7587e7a0b27"}
Jul 28 14:19:30.531378 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:30.531329    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-br86x" podStartSLOduration=1.556416952 podStartE2EDuration="5.531312001s" totalImagesPullingTime="3.974895049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:30.530639239 +0000 UTC m=+40.818500181" watchObservedRunningTime="2026-07-28 14:19:30.531312001 +0000 UTC m=+40.819172946"
Jul 28 14:19:30.583823 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:30.583772    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8dm86" podStartSLOduration=8.231356188 podStartE2EDuration="40.583758069s" totalImagesPullingTime="32.352401881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:30.582055285 +0000 UTC m=+40.869916228" watchObservedRunningTime="2026-07-28 14:19:30.583758069 +0000 UTC m=+40.871619012"
Jul 28 14:19:32.244585 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.244552    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-gl246"]
Jul 28 14:19:32.277347 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.277322    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-gl246"]
Jul 28 14:19:32.277463 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.277409    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.280644 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.280620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 14:19:32.280644 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.280640    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 14:19:32.280805 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.280668    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xw8mk\""
Jul 28 14:19:32.282114 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.282099    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:19:32.282174 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.282125    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 14:19:32.287490 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.287465    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 14:19:32.345497 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.345477    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"]
Jul 28 14:19:32.354481 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.354458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h8rm\" (UniqueName: \"kubernetes.io/projected/3513d484-71fc-4da3-8021-4260e01b51b1-kube-api-access-8h8rm\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.354588 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.354488    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3513d484-71fc-4da3-8021-4260e01b51b1-config\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.354588 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.354513    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3513d484-71fc-4da3-8021-4260e01b51b1-trusted-ca\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.354588 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.354530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3513d484-71fc-4da3-8021-4260e01b51b1-serving-cert\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.367065 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.367047    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"]
Jul 28 14:19:32.367191 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.367175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.370140 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.370122    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 14:19:32.370248 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.370171    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 14:19:32.370311 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.370261    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 14:19:32.371023 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.371007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7tgcc\""
Jul 28 14:19:32.371468 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.371450    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:19:32.391167 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.391144    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"]
Jul 28 14:19:32.391303 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.391175    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"]
Jul 28 14:19:32.391303 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.391268    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:32.394252 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.394227    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-252p9\""
Jul 28 14:19:32.394373 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.394284    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 14:19:32.394373 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.394291    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 14:19:32.394373 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.394289    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:19:32.455267 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3513d484-71fc-4da3-8021-4260e01b51b1-serving-cert\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.455451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e9d31a5-b09b-4416-a824-4c049c13d215-config\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.455451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9d31a5-b09b-4416-a824-4c049c13d215-serving-cert\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.455578 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h8rm\" (UniqueName: \"kubernetes.io/projected/3513d484-71fc-4da3-8021-4260e01b51b1-kube-api-access-8h8rm\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.455578 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6rl2\" (UniqueName: \"kubernetes.io/projected/0e9d31a5-b09b-4416-a824-4c049c13d215-kube-api-access-j6rl2\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.455578 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3513d484-71fc-4da3-8021-4260e01b51b1-config\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.455578 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:32.455747 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p25r\" (UniqueName: \"kubernetes.io/projected/5474c745-d7b3-4d58-8030-69ba393494fb-kube-api-access-4p25r\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:32.455802 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.455743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3513d484-71fc-4da3-8021-4260e01b51b1-trusted-ca\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.456239 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.456204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3513d484-71fc-4da3-8021-4260e01b51b1-config\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.456467 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.456450    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3513d484-71fc-4da3-8021-4260e01b51b1-trusted-ca\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.459044 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.459026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3513d484-71fc-4da3-8021-4260e01b51b1-serving-cert\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.465286 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.465258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h8rm\" (UniqueName: \"kubernetes.io/projected/3513d484-71fc-4da3-8021-4260e01b51b1-kube-api-access-8h8rm\") pod \"console-operator-584dcb6b54-gl246\" (UID: \"3513d484-71fc-4da3-8021-4260e01b51b1\") " pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.556037 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.555976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6rl2\" (UniqueName: \"kubernetes.io/projected/0e9d31a5-b09b-4416-a824-4c049c13d215-kube-api-access-j6rl2\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.556037 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.556006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:32.556037 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.556024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p25r\" (UniqueName: \"kubernetes.io/projected/5474c745-d7b3-4d58-8030-69ba393494fb-kube-api-access-4p25r\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:32.556235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.556092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e9d31a5-b09b-4416-a824-4c049c13d215-config\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.556235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.556133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9d31a5-b09b-4416-a824-4c049c13d215-serving-cert\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.556235 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:32.556227    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:19:32.556329 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:32.556302    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls podName:5474c745-d7b3-4d58-8030-69ba393494fb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:33.056287132 +0000 UTC m=+43.344148073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls") pod "cluster-samples-operator-6748489656-dtfw5" (UID: "5474c745-d7b3-4d58-8030-69ba393494fb") : secret "samples-operator-tls" not found
Jul 28 14:19:32.556659 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.556627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e9d31a5-b09b-4416-a824-4c049c13d215-config\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.558081 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.558063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9d31a5-b09b-4416-a824-4c049c13d215-serving-cert\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.566459 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.566439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6rl2\" (UniqueName: \"kubernetes.io/projected/0e9d31a5-b09b-4416-a824-4c049c13d215-kube-api-access-j6rl2\") pod \"service-ca-operator-5d6c75f84f-7gbjm\" (UID: \"0e9d31a5-b09b-4416-a824-4c049c13d215\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.566657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.566639    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p25r\" (UniqueName: \"kubernetes.io/projected/5474c745-d7b3-4d58-8030-69ba393494fb-kube-api-access-4p25r\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:32.586470 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.586449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:32.677040 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.676795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"
Jul 28 14:19:32.764271 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.764131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-gl246"]
Jul 28 14:19:32.767059 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:32.767034    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3513d484_71fc_4da3_8021_4260e01b51b1.slice/crio-59bfa93d3b56c899a12b51ea3ab4820b22fd7f3441b115bd4915267f176c88f0 WatchSource:0}: Error finding container 59bfa93d3b56c899a12b51ea3ab4820b22fd7f3441b115bd4915267f176c88f0: Status 404 returned error can't find the container with id 59bfa93d3b56c899a12b51ea3ab4820b22fd7f3441b115bd4915267f176c88f0
Jul 28 14:19:32.819128 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.819011    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm"]
Jul 28 14:19:32.821320 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:32.821289    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9d31a5_b09b_4416_a824_4c049c13d215.slice/crio-57404e2a73413a7122f51a19448b9a45e7302317f66a6ddfe621de40ff8032ff WatchSource:0}: Error finding container 57404e2a73413a7122f51a19448b9a45e7302317f66a6ddfe621de40ff8032ff: Status 404 returned error can't find the container with id 57404e2a73413a7122f51a19448b9a45e7302317f66a6ddfe621de40ff8032ff
Jul 28 14:19:32.959135 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:32.959107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:32.959269 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:32.959252    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:32.959320 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:32.959308    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls podName:9bcedd50-8330-4d98-98c7-1df1500b80bb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:40.959293777 +0000 UTC m=+51.247154697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls") pod "dns-default-hxz4r" (UID: "9bcedd50-8330-4d98-98c7-1df1500b80bb") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:33.059756 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:33.059717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:33.059866 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:33.059851    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:19:33.059917 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:33.059907    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls podName:5474c745-d7b3-4d58-8030-69ba393494fb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:34.05989006 +0000 UTC m=+44.347750980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls") pod "cluster-samples-operator-6748489656-dtfw5" (UID: "5474c745-d7b3-4d58-8030-69ba393494fb") : secret "samples-operator-tls" not found
Jul 28 14:19:33.513092 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:33.513057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm" event={"ID":"0e9d31a5-b09b-4416-a824-4c049c13d215","Type":"ContainerStarted","Data":"57404e2a73413a7122f51a19448b9a45e7302317f66a6ddfe621de40ff8032ff"}
Jul 28 14:19:33.514162 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:33.514138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" event={"ID":"3513d484-71fc-4da3-8021-4260e01b51b1","Type":"ContainerStarted","Data":"59bfa93d3b56c899a12b51ea3ab4820b22fd7f3441b115bd4915267f176c88f0"}
Jul 28 14:19:34.066107 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:34.066071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:34.066313 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:34.066248    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:19:34.066383 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:34.066325    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls podName:5474c745-d7b3-4d58-8030-69ba393494fb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:36.066304837 +0000 UTC m=+46.354165762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls") pod "cluster-samples-operator-6748489656-dtfw5" (UID: "5474c745-d7b3-4d58-8030-69ba393494fb") : secret "samples-operator-tls" not found
Jul 28 14:19:34.468345 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:34.468302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:34.468520 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:34.468419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:34.468520 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:34.468461    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:34.468593 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:34.468527    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 14:19:34.468593 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:34.468534    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:42.468514941 +0000 UTC m=+52.756375864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:34.468593 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:34.468565    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:42.468554823 +0000 UTC m=+52.756415752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : configmap "kube-rbac-proxy" not found
Jul 28 14:19:36.080801 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:36.080776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:36.081145 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:36.080917    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:19:36.081145 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:36.080970    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls podName:5474c745-d7b3-4d58-8030-69ba393494fb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:40.080955364 +0000 UTC m=+50.368816285 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls") pod "cluster-samples-operator-6748489656-dtfw5" (UID: "5474c745-d7b3-4d58-8030-69ba393494fb") : secret "samples-operator-tls" not found
Jul 28 14:19:36.523460 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:36.523419    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm" event={"ID":"0e9d31a5-b09b-4416-a824-4c049c13d215","Type":"ContainerStarted","Data":"5c8a755e0d35adfe537155a54b884badbdb8955342393bc3349603f35337b284"}
Jul 28 14:19:36.525052 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:36.525032    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/0.log"
Jul 28 14:19:36.525167 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:36.525067    2579 generic.go:356] "Generic (PLEG): container finished" podID="3513d484-71fc-4da3-8021-4260e01b51b1" containerID="e3903122b571d250be396a37daacb0f539297093cc2454afcde14f7249e9c4c6" exitCode=255
Jul 28 14:19:36.525167 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:36.525093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" event={"ID":"3513d484-71fc-4da3-8021-4260e01b51b1","Type":"ContainerDied","Data":"e3903122b571d250be396a37daacb0f539297093cc2454afcde14f7249e9c4c6"}
Jul 28 14:19:36.525697 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:36.525681    2579 scope.go:117] "RemoveContainer" containerID="e3903122b571d250be396a37daacb0f539297093cc2454afcde14f7249e9c4c6"
Jul 28 14:19:36.543162 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:36.541750    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm" podStartSLOduration=1.290850077 podStartE2EDuration="4.541737188s" totalImagesPullingTime="3.250887111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:36.540480029 +0000 UTC m=+46.828340974" watchObservedRunningTime="2026-07-28 14:19:36.541737188 +0000 UTC m=+46.829598132"
Jul 28 14:19:37.528934 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:37.528906    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:19:37.529358 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:37.529289    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/0.log"
Jul 28 14:19:37.529358 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:37.529322    2579 generic.go:356] "Generic (PLEG): container finished" podID="3513d484-71fc-4da3-8021-4260e01b51b1" containerID="85d3f7536d0b3b7006b5af5c0a0cd62f051b5ace8945ea52fc8b5fd12740c51b" exitCode=255
Jul 28 14:19:37.529427 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:37.529385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" event={"ID":"3513d484-71fc-4da3-8021-4260e01b51b1","Type":"ContainerDied","Data":"85d3f7536d0b3b7006b5af5c0a0cd62f051b5ace8945ea52fc8b5fd12740c51b"}
Jul 28 14:19:37.529467 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:37.529456    2579 scope.go:117] "RemoveContainer" containerID="e3903122b571d250be396a37daacb0f539297093cc2454afcde14f7249e9c4c6"
Jul 28 14:19:37.529638 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:37.529613    2579 scope.go:117] "RemoveContainer" containerID="85d3f7536d0b3b7006b5af5c0a0cd62f051b5ace8945ea52fc8b5fd12740c51b"
Jul 28 14:19:37.529806 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:37.529783    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-gl246_openshift-console-operator(3513d484-71fc-4da3-8021-4260e01b51b1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" podUID="3513d484-71fc-4da3-8021-4260e01b51b1"
Jul 28 14:19:38.533266 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:38.533236    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:19:38.533643 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:38.533625    2579 scope.go:117] "RemoveContainer" containerID="85d3f7536d0b3b7006b5af5c0a0cd62f051b5ace8945ea52fc8b5fd12740c51b"
Jul 28 14:19:38.533833 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:38.533810    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-gl246_openshift-console-operator(3513d484-71fc-4da3-8021-4260e01b51b1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" podUID="3513d484-71fc-4da3-8021-4260e01b51b1"
Jul 28 14:19:39.535072 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.535034    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-g95x4"]
Jul 28 14:19:39.568491 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.568467    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-g95x4"]
Jul 28 14:19:39.568614 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.568554    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.571707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.571687    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 14:19:39.572004 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.571987    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qx9z4\""
Jul 28 14:19:39.573086 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.573068    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 14:19:39.573166 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.573108    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 14:19:39.573166 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.573131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 14:19:39.607600 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.607579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9rcw\" (UniqueName: \"kubernetes.io/projected/d5951654-29dd-4535-8a1e-0d93d988b926-kube-api-access-v9rcw\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.607675 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.607620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d5951654-29dd-4535-8a1e-0d93d988b926-signing-key\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.607724 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.607703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d5951654-29dd-4535-8a1e-0d93d988b926-signing-cabundle\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.708766 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.708746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d5951654-29dd-4535-8a1e-0d93d988b926-signing-cabundle\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.708863 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.708811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9rcw\" (UniqueName: \"kubernetes.io/projected/d5951654-29dd-4535-8a1e-0d93d988b926-kube-api-access-v9rcw\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.708863 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.708854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d5951654-29dd-4535-8a1e-0d93d988b926-signing-key\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.709375 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.709349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d5951654-29dd-4535-8a1e-0d93d988b926-signing-cabundle\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.711049 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.711023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d5951654-29dd-4535-8a1e-0d93d988b926-signing-key\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.718061 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.718037    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9rcw\" (UniqueName: \"kubernetes.io/projected/d5951654-29dd-4535-8a1e-0d93d988b926-kube-api-access-v9rcw\") pod \"service-ca-658d7b7c78-g95x4\" (UID: \"d5951654-29dd-4535-8a1e-0d93d988b926\") " pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:39.877518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:39.877455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-g95x4"
Jul 28 14:19:40.016380 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:40.016353    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-g95x4"]
Jul 28 14:19:40.018650 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:40.018627    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5951654_29dd_4535_8a1e_0d93d988b926.slice/crio-512df85d60ebb6233b42e2895c6e2b71948884517ca6d8603fb83b8d12fc3a17 WatchSource:0}: Error finding container 512df85d60ebb6233b42e2895c6e2b71948884517ca6d8603fb83b8d12fc3a17: Status 404 returned error can't find the container with id 512df85d60ebb6233b42e2895c6e2b71948884517ca6d8603fb83b8d12fc3a17
Jul 28 14:19:40.112174 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:40.112144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:40.112348 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:40.112294    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 14:19:40.112393 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:40.112364    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls podName:5474c745-d7b3-4d58-8030-69ba393494fb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:48.11234888 +0000 UTC m=+58.400209805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls") pod "cluster-samples-operator-6748489656-dtfw5" (UID: "5474c745-d7b3-4d58-8030-69ba393494fb") : secret "samples-operator-tls" not found
Jul 28 14:19:40.539965 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:40.539931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-g95x4" event={"ID":"d5951654-29dd-4535-8a1e-0d93d988b926","Type":"ContainerStarted","Data":"863de6d0df048984509d28ce8f423fcacef9a17708b277bf896bc6e9c89138dc"}
Jul 28 14:19:40.540331 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:40.539984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-g95x4" event={"ID":"d5951654-29dd-4535-8a1e-0d93d988b926","Type":"ContainerStarted","Data":"512df85d60ebb6233b42e2895c6e2b71948884517ca6d8603fb83b8d12fc3a17"}
Jul 28 14:19:40.557928 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:40.557885    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-g95x4" podStartSLOduration=1.557872379 podStartE2EDuration="1.557872379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:40.556971337 +0000 UTC m=+50.844832281" watchObservedRunningTime="2026-07-28 14:19:40.557872379 +0000 UTC m=+50.845733321"
Jul 28 14:19:41.019936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:41.019901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:41.020093 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:41.020051    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 14:19:41.020135 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:41.020121    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls podName:9bcedd50-8330-4d98-98c7-1df1500b80bb nodeName:}" failed. No retries permitted until 2026-07-28 14:19:57.020106745 +0000 UTC m=+67.307967666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls") pod "dns-default-hxz4r" (UID: "9bcedd50-8330-4d98-98c7-1df1500b80bb") : secret "dns-default-metrics-tls" not found
Jul 28 14:19:42.533463 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:42.533432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:42.533994 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:42.533498    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:42.533994 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:42.533600    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:42.533994 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:42.533681    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls podName:307891dc-8150-4d47-8fa3-849b4b25c3dc nodeName:}" failed. No retries permitted until 2026-07-28 14:19:58.533658998 +0000 UTC m=+68.821519923 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8sngc" (UID: "307891dc-8150-4d47-8fa3-849b4b25c3dc") : secret "insights-runtime-extractor-tls" not found
Jul 28 14:19:42.533994 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:42.533905    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/307891dc-8150-4d47-8fa3-849b4b25c3dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:42.587575 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:42.587550    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:42.587713 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:42.587666    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:42.587947 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:42.587933    2579 scope.go:117] "RemoveContainer" containerID="85d3f7536d0b3b7006b5af5c0a0cd62f051b5ace8945ea52fc8b5fd12740c51b"
Jul 28 14:19:42.588108 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:42.588089    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-gl246_openshift-console-operator(3513d484-71fc-4da3-8021-4260e01b51b1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" podUID="3513d484-71fc-4da3-8021-4260e01b51b1"
Jul 28 14:19:43.548784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:43.548753    2579 scope.go:117] "RemoveContainer" containerID="85d3f7536d0b3b7006b5af5c0a0cd62f051b5ace8945ea52fc8b5fd12740c51b"
Jul 28 14:19:43.549314 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:43.548893    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-gl246_openshift-console-operator(3513d484-71fc-4da3-8021-4260e01b51b1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" podUID="3513d484-71fc-4da3-8021-4260e01b51b1"
Jul 28 14:19:48.177346 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:48.177311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:48.179642 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:48.179619    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5474c745-d7b3-4d58-8030-69ba393494fb-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-dtfw5\" (UID: \"5474c745-d7b3-4d58-8030-69ba393494fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:48.299492 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:48.299467    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"
Jul 28 14:19:48.420097 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:48.420060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5"]
Jul 28 14:19:48.561784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:48.561751    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5" event={"ID":"5474c745-d7b3-4d58-8030-69ba393494fb","Type":"ContainerStarted","Data":"75b8dfe52768ca9247d35ee837e41372a2ad2dedb97b4a6927bb46f55722cfdd"}
Jul 28 14:19:51.475378 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:51.475348    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qgrn"
Jul 28 14:19:51.576062 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:51.576030    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5" event={"ID":"5474c745-d7b3-4d58-8030-69ba393494fb","Type":"ContainerStarted","Data":"f9f107314aee98e3dd7609a8986040cb462c721c3f32d5f1768c8949c76a5296"}
Jul 28 14:19:51.576235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:51.576105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5" event={"ID":"5474c745-d7b3-4d58-8030-69ba393494fb","Type":"ContainerStarted","Data":"8a73f684129f9c6e91241b56f57c8b3930428187ed93fc46c7c33ecb2a0ba524"}
Jul 28 14:19:51.595266 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:51.595201    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-dtfw5" podStartSLOduration=17.40824512 podStartE2EDuration="19.595189356s" totalImagesPullingTime="2.186944236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:51.59475663 +0000 UTC m=+61.882617604" watchObservedRunningTime="2026-07-28 14:19:51.595189356 +0000 UTC m=+61.883050299"
Jul 28 14:19:54.186751 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:54.186722    2579 scope.go:117] "RemoveContainer" containerID="85d3f7536d0b3b7006b5af5c0a0cd62f051b5ace8945ea52fc8b5fd12740c51b"
Jul 28 14:19:54.586831 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:54.586752    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:19:54.586831 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:54.586821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" event={"ID":"3513d484-71fc-4da3-8021-4260e01b51b1","Type":"ContainerStarted","Data":"53e109885a2a0ca8d26a68903fd6b4bb2d6d5953f90c6142b20771fefb08b0e5"}
Jul 28 14:19:54.587152 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:54.587126    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:54.594394 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:54.594370    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-gl246"
Jul 28 14:19:54.626849 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:54.626793    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-gl246" podStartSLOduration=19.31686496 podStartE2EDuration="22.626777349s" totalImagesPullingTime="3.309912389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:54.609023401 +0000 UTC m=+64.896884367" watchObservedRunningTime="2026-07-28 14:19:54.626777349 +0000 UTC m=+64.914638270"
Jul 28 14:19:55.838774 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.838731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:55.842520 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.842500    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 14:19:55.851953 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.851930    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 14:19:55.861698 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.861676    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2vbb\" (UniqueName: \"kubernetes.io/projected/46c7fba6-aaa2-4a69-9782-d2becb210717-kube-api-access-w2vbb\") pod \"network-check-target-9wstk\" (UID: \"46c7fba6-aaa2-4a69-9782-d2becb210717\") " pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:55.939755 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.939717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:55.939879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.939769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:55.942983 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.942962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 14:19:55.943053 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.942995    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 14:19:55.952673 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.952647    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6-original-pull-secret\") pod \"global-pull-secret-syncer-d6pqw\" (UID: \"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6\") " pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:55.952673 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:55.952668    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe53c95f-e6ee-4420-bbd9-4a27000d6161-metrics-certs\") pod \"network-metrics-daemon-xz6w2\" (UID: \"fe53c95f-e6ee-4420-bbd9-4a27000d6161\") " pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:56.098408 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.098344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6pqw"
Jul 28 14:19:56.106274 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.106247    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v4vsp\""
Jul 28 14:19:56.113600 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.113581    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:19:56.201372 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.201351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gd6j\""
Jul 28 14:19:56.208922 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.208880    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xz6w2"
Jul 28 14:19:56.227672 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.227643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d6pqw"]
Jul 28 14:19:56.231690 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:56.231666    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f352d50_ad34_4f97_b7d6_3e3d9cdf31d6.slice/crio-016003cbe54140e610b1eadf606057eee54763e1cea493b6403c250adc0062e5 WatchSource:0}: Error finding container 016003cbe54140e610b1eadf606057eee54763e1cea493b6403c250adc0062e5: Status 404 returned error can't find the container with id 016003cbe54140e610b1eadf606057eee54763e1cea493b6403c250adc0062e5
Jul 28 14:19:56.245661 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.245641    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9wstk"]
Jul 28 14:19:56.247843 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:56.247823    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c7fba6_aaa2_4a69_9782_d2becb210717.slice/crio-97e585be1c2e4d04faef294b66c5a034c793a01f41bedd3d9ee88f6d34b9e66e WatchSource:0}: Error finding container 97e585be1c2e4d04faef294b66c5a034c793a01f41bedd3d9ee88f6d34b9e66e: Status 404 returned error can't find the container with id 97e585be1c2e4d04faef294b66c5a034c793a01f41bedd3d9ee88f6d34b9e66e
Jul 28 14:19:56.334635 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.334606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xz6w2"]
Jul 28 14:19:56.337451 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:56.337421    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe53c95f_e6ee_4420_bbd9_4a27000d6161.slice/crio-93bae3659d46d6d30954b06bf8e5463bd7462d39d0d693380508b025865873be WatchSource:0}: Error finding container 93bae3659d46d6d30954b06bf8e5463bd7462d39d0d693380508b025865873be: Status 404 returned error can't find the container with id 93bae3659d46d6d30954b06bf8e5463bd7462d39d0d693380508b025865873be
Jul 28 14:19:56.596983 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.596948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xz6w2" event={"ID":"fe53c95f-e6ee-4420-bbd9-4a27000d6161","Type":"ContainerStarted","Data":"93bae3659d46d6d30954b06bf8e5463bd7462d39d0d693380508b025865873be"}
Jul 28 14:19:56.598032 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.598005    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9wstk" event={"ID":"46c7fba6-aaa2-4a69-9782-d2becb210717","Type":"ContainerStarted","Data":"97e585be1c2e4d04faef294b66c5a034c793a01f41bedd3d9ee88f6d34b9e66e"}
Jul 28 14:19:56.599001 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:56.598979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d6pqw" event={"ID":"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6","Type":"ContainerStarted","Data":"016003cbe54140e610b1eadf606057eee54763e1cea493b6403c250adc0062e5"}
Jul 28 14:19:57.048421 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:57.048317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:57.052434 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:57.052378    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bcedd50-8330-4d98-98c7-1df1500b80bb-metrics-tls\") pod \"dns-default-hxz4r\" (UID: \"9bcedd50-8330-4d98-98c7-1df1500b80bb\") " pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:57.242742 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:57.242714    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6f28j\""
Jul 28 14:19:57.249795 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:57.249762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:19:57.640669 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:57.640637    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hxz4r"]
Jul 28 14:19:57.647358 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:19:57.647322    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bcedd50_8330_4d98_98c7_1df1500b80bb.slice/crio-489225f229b38ff25de23d2259c52583065375a6332a48f1d0f3f7963d1c8975 WatchSource:0}: Error finding container 489225f229b38ff25de23d2259c52583065375a6332a48f1d0f3f7963d1c8975: Status 404 returned error can't find the container with id 489225f229b38ff25de23d2259c52583065375a6332a48f1d0f3f7963d1c8975
Jul 28 14:19:58.560939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.560905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:58.563482 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.563454    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/307891dc-8150-4d47-8fa3-849b4b25c3dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8sngc\" (UID: \"307891dc-8150-4d47-8fa3-849b4b25c3dc\") " pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:58.609643 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.609609    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xz6w2" event={"ID":"fe53c95f-e6ee-4420-bbd9-4a27000d6161","Type":"ContainerStarted","Data":"7c3c510cf85fe96912b90f9d00064c0e4068dcc413c2f2984906e7e8be6d1e76"}
Jul 28 14:19:58.609792 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.609666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xz6w2" event={"ID":"fe53c95f-e6ee-4420-bbd9-4a27000d6161","Type":"ContainerStarted","Data":"dd933058242fa30a3bc1738cc0f63c4de9e1cf65e5e9d78c46a41a90cb5f2b14"}
Jul 28 14:19:58.610963 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.610926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hxz4r" event={"ID":"9bcedd50-8330-4d98-98c7-1df1500b80bb","Type":"ContainerStarted","Data":"489225f229b38ff25de23d2259c52583065375a6332a48f1d0f3f7963d1c8975"}
Jul 28 14:19:58.627639 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.627583    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xz6w2" podStartSLOduration=67.113640912 podStartE2EDuration="1m8.6274282s" totalImagesPullingTime="1.513787288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:19:58.626995651 +0000 UTC m=+68.914856627" watchObservedRunningTime="2026-07-28 14:19:58.6274282 +0000 UTC m=+68.915289145"
Jul 28 14:19:58.786258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.786226    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kr6cm\""
Jul 28 14:19:58.793244 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:58.793104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8sngc"
Jul 28 14:19:59.517163 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.517130    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"]
Jul 28 14:19:59.533682 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.533654    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"]
Jul 28 14:19:59.533821 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.533703    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.537838 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.537816    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 14:19:59.539132 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.539112    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-nc2q5\""
Jul 28 14:19:59.539260 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.539111    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 14:19:59.539411 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.539396    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 14:19:59.539477 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.539418    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 14:19:59.631385 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.631353    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f9784dbbc-k6ccg"]
Jul 28 14:19:59.643231 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.643155    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.646692 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.646665    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 14:19:59.647915 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.647893    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 14:19:59.648235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.648134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lj7bw\""
Jul 28 14:19:59.648575 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.648554    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 14:19:59.649627 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.649606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f9784dbbc-k6ccg"]
Jul 28 14:19:59.654789 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.654770    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 14:19:59.670354 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.670330    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61937a1d-a055-4c37-bf04-925fce5591f6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7df746cf9d-skw8r\" (UID: \"61937a1d-a055-4c37-bf04-925fce5591f6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.670633 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.670610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r7l2\" (UniqueName: \"kubernetes.io/projected/61937a1d-a055-4c37-bf04-925fce5591f6-kube-api-access-6r7l2\") pod \"managed-serviceaccount-addon-agent-7df746cf9d-skw8r\" (UID: \"61937a1d-a055-4c37-bf04-925fce5591f6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.683813 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.683788    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f9784dbbc-k6ccg"]
Jul 28 14:19:59.684028 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:19:59.684002    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-9fcd4 registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-9fcd4 registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg" podUID="e50a4232-e83a-431c-ba96-235d9f7efbd6"
Jul 28 14:19:59.732885 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.732846    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-brnr7"]
Jul 28 14:19:59.747157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.747137    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-brnr7"]
Jul 28 14:19:59.747313 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.747251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-brnr7"
Jul 28 14:19:59.750734 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.750674    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 14:19:59.750866 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.750791    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 14:19:59.750981 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.750945    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ddbg9\""
Jul 28 14:19:59.771009 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.770948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-installation-pull-secrets\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.771009 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.770984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61937a1d-a055-4c37-bf04-925fce5591f6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7df746cf9d-skw8r\" (UID: \"61937a1d-a055-4c37-bf04-925fce5591f6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.771161 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r7l2\" (UniqueName: \"kubernetes.io/projected/61937a1d-a055-4c37-bf04-925fce5591f6-kube-api-access-6r7l2\") pod \"managed-serviceaccount-addon-agent-7df746cf9d-skw8r\" (UID: \"61937a1d-a055-4c37-bf04-925fce5591f6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.771161 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-image-registry-private-configuration\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.771161 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-trusted-ca\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.771161 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-certificates\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.771400 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e50a4232-e83a-431c-ba96-235d9f7efbd6-ca-trust-extracted\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.771400 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fcd4\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-kube-api-access-9fcd4\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.771400 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-bound-sa-token\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.771400 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.771344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-tls\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.773603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.773580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61937a1d-a055-4c37-bf04-925fce5591f6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7df746cf9d-skw8r\" (UID: \"61937a1d-a055-4c37-bf04-925fce5591f6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.784550 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.784523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r7l2\" (UniqueName: \"kubernetes.io/projected/61937a1d-a055-4c37-bf04-925fce5591f6-kube-api-access-6r7l2\") pod \"managed-serviceaccount-addon-agent-7df746cf9d-skw8r\" (UID: \"61937a1d-a055-4c37-bf04-925fce5591f6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.850141 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.850107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"
Jul 28 14:19:59.872670 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.872636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-image-registry-private-configuration\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.872800 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.872772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-trusted-ca\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.872860 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.872816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-certificates\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.872860 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.872846    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e50a4232-e83a-431c-ba96-235d9f7efbd6-ca-trust-extracted\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.873114 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.873095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fcd4\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-kube-api-access-9fcd4\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.873186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.873138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-bound-sa-token\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.873186 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.873170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b8nm\" (UniqueName: \"kubernetes.io/projected/beb56aef-8622-474e-85b4-7a7d265fca68-kube-api-access-7b8nm\") pod \"downloads-7c97dc4555-brnr7\" (UID: \"beb56aef-8622-474e-85b4-7a7d265fca68\") " pod="openshift-console/downloads-7c97dc4555-brnr7"
Jul 28 14:19:59.873314 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.873208    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-tls\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.873314 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.873259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-installation-pull-secrets\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.873754 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.873206    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e50a4232-e83a-431c-ba96-235d9f7efbd6-ca-trust-extracted\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.873848 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.873787    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-trusted-ca\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.874506 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.874464    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-certificates\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.875995 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.875960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-image-registry-private-configuration\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.876448 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.876428    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-installation-pull-secrets\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.876573 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.876553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-tls\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.889055 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.889034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fcd4\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-kube-api-access-9fcd4\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.889636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.889615    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-bound-sa-token\") pod \"image-registry-f9784dbbc-k6ccg\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") " pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:19:59.974539 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.974478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b8nm\" (UniqueName: \"kubernetes.io/projected/beb56aef-8622-474e-85b4-7a7d265fca68-kube-api-access-7b8nm\") pod \"downloads-7c97dc4555-brnr7\" (UID: \"beb56aef-8622-474e-85b4-7a7d265fca68\") " pod="openshift-console/downloads-7c97dc4555-brnr7"
Jul 28 14:19:59.985875 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:19:59.985850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b8nm\" (UniqueName: \"kubernetes.io/projected/beb56aef-8622-474e-85b4-7a7d265fca68-kube-api-access-7b8nm\") pod \"downloads-7c97dc4555-brnr7\" (UID: \"beb56aef-8622-474e-85b4-7a7d265fca68\") " pod="openshift-console/downloads-7c97dc4555-brnr7"
Jul 28 14:20:00.058677 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.058591    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-brnr7"
Jul 28 14:20:00.621919 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.621885    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:20:00.628940 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.628917    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:20:00.681778 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681746    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-installation-pull-secrets\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682224 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fcd4\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-kube-api-access-9fcd4\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682224 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681829    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-certificates\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682224 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681853    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-tls\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682224 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681887    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-trusted-ca\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682224 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681911    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-bound-sa-token\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682224 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681956    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-image-registry-private-configuration\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682224 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.681987    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e50a4232-e83a-431c-ba96-235d9f7efbd6-ca-trust-extracted\") pod \"e50a4232-e83a-431c-ba96-235d9f7efbd6\" (UID: \"e50a4232-e83a-431c-ba96-235d9f7efbd6\") "
Jul 28 14:20:00.682580 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.682362    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e50a4232-e83a-431c-ba96-235d9f7efbd6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:20:00.683013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.682966    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:20:00.683425 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.683369    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:20:00.684437 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.684411    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-kube-api-access-9fcd4" (OuterVolumeSpecName: "kube-api-access-9fcd4") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "kube-api-access-9fcd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:20:00.684546 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.684432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:20:00.684636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.684615    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:20:00.684821 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.684803    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:20:00.685077 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.685059    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e50a4232-e83a-431c-ba96-235d9f7efbd6" (UID: "e50a4232-e83a-431c-ba96-235d9f7efbd6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:20:00.782660 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782631    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-image-registry-private-configuration\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:00.782780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782665    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e50a4232-e83a-431c-ba96-235d9f7efbd6-ca-trust-extracted\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:00.782780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782681    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e50a4232-e83a-431c-ba96-235d9f7efbd6-installation-pull-secrets\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:00.782780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782696    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fcd4\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-kube-api-access-9fcd4\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:00.782780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782714    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-certificates\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:00.782780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782727    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-registry-tls\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:00.782780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782739    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e50a4232-e83a-431c-ba96-235d9f7efbd6-trusted-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:00.782780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:00.782753    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e50a4232-e83a-431c-ba96-235d9f7efbd6-bound-sa-token\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:01.633274 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:01.631514    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9784dbbc-k6ccg"
Jul 28 14:20:01.680183 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:01.680126    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f9784dbbc-k6ccg"]
Jul 28 14:20:01.686101 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:01.686013    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f9784dbbc-k6ccg"]
Jul 28 14:20:01.707235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:01.707105    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8sngc"]
Jul 28 14:20:01.711407 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:01.711367    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307891dc_8150_4d47_8fa3_849b4b25c3dc.slice/crio-0db4ea43ecde4b594e7272e31c3189c4b4732b1cf12595fbe1fd1ad314074bde WatchSource:0}: Error finding container 0db4ea43ecde4b594e7272e31c3189c4b4732b1cf12595fbe1fd1ad314074bde: Status 404 returned error can't find the container with id 0db4ea43ecde4b594e7272e31c3189c4b4732b1cf12595fbe1fd1ad314074bde
Jul 28 14:20:01.721958 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:01.721942    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-brnr7"]
Jul 28 14:20:01.724311 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:01.724282    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb56aef_8622_474e_85b4_7a7d265fca68.slice/crio-c48c5c3bf9cfb744df6b71cb4f94179e579eff2c7debd3bc963ef5ac8baf6df5 WatchSource:0}: Error finding container c48c5c3bf9cfb744df6b71cb4f94179e579eff2c7debd3bc963ef5ac8baf6df5: Status 404 returned error can't find the container with id c48c5c3bf9cfb744df6b71cb4f94179e579eff2c7debd3bc963ef5ac8baf6df5
Jul 28 14:20:01.742709 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:01.742680    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r"]
Jul 28 14:20:01.747042 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:01.746880    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61937a1d_a055_4c37_bf04_925fce5591f6.slice/crio-d7095efc1c96766ec731841eeea7396219debd2ff65ab2060e0c073114ccc0e0 WatchSource:0}: Error finding container d7095efc1c96766ec731841eeea7396219debd2ff65ab2060e0c073114ccc0e0: Status 404 returned error can't find the container with id d7095efc1c96766ec731841eeea7396219debd2ff65ab2060e0c073114ccc0e0
Jul 28 14:20:02.190202 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.190172    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e50a4232-e83a-431c-ba96-235d9f7efbd6" path="/var/lib/kubelet/pods/e50a4232-e83a-431c-ba96-235d9f7efbd6/volumes"
Jul 28 14:20:02.639911 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.639017    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9wstk" event={"ID":"46c7fba6-aaa2-4a69-9782-d2becb210717","Type":"ContainerStarted","Data":"554dc05873f1ab5731605d017cff86b5a8439011a7ed08c19a32e00d45d259db"}
Jul 28 14:20:02.639911 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.639807    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:20:02.643173 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.643145    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hxz4r" event={"ID":"9bcedd50-8330-4d98-98c7-1df1500b80bb","Type":"ContainerStarted","Data":"d3562bd5d794eba67c7f0c55697dcdc428fcc8d65f8a8c53c2cf976f5683d75b"}
Jul 28 14:20:02.643314 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.643195    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hxz4r" event={"ID":"9bcedd50-8330-4d98-98c7-1df1500b80bb","Type":"ContainerStarted","Data":"0e432a98877692d2b2519c133295d81d18c589cedd2c9efa6f55e6c1cadb5c13"}
Jul 28 14:20:02.643840 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.643721    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:20:02.647468 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.647401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-brnr7" event={"ID":"beb56aef-8622-474e-85b4-7a7d265fca68","Type":"ContainerStarted","Data":"c48c5c3bf9cfb744df6b71cb4f94179e579eff2c7debd3bc963ef5ac8baf6df5"}
Jul 28 14:20:02.649599 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.649548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8sngc" event={"ID":"307891dc-8150-4d47-8fa3-849b4b25c3dc","Type":"ContainerStarted","Data":"0db4ea43ecde4b594e7272e31c3189c4b4732b1cf12595fbe1fd1ad314074bde"}
Jul 28 14:20:02.651563 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.651532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r" event={"ID":"61937a1d-a055-4c37-bf04-925fce5591f6","Type":"ContainerStarted","Data":"d7095efc1c96766ec731841eeea7396219debd2ff65ab2060e0c073114ccc0e0"}
Jul 28 14:20:02.654597 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.654185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d6pqw" event={"ID":"2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6","Type":"ContainerStarted","Data":"0ecca5edc1872e46ac4a48bae3b4350945ddb5e10422b9330e036f9cae3d5ecd"}
Jul 28 14:20:02.676625 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.676578    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d6pqw" podStartSLOduration=66.338588186 podStartE2EDuration="1m11.676559409s" totalImagesPullingTime="5.337971223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:02.676514167 +0000 UTC m=+72.964375112" watchObservedRunningTime="2026-07-28 14:20:02.676559409 +0000 UTC m=+72.964420354"
Jul 28 14:20:02.677252 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.677121    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9wstk" podStartSLOduration=67.394070289 podStartE2EDuration="1m12.677110819s" totalImagesPullingTime="5.28304053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:02.658766945 +0000 UTC m=+72.946627906" watchObservedRunningTime="2026-07-28 14:20:02.677110819 +0000 UTC m=+72.964971764"
Jul 28 14:20:02.697584 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:02.697538    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hxz4r" podStartSLOduration=33.761950412 podStartE2EDuration="37.697519915s" totalImagesPullingTime="3.935569503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:02.697289101 +0000 UTC m=+72.985150063" watchObservedRunningTime="2026-07-28 14:20:02.697519915 +0000 UTC m=+72.985380859"
Jul 28 14:20:03.678052 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.678023    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6587ff64d9-m9s66"]
Jul 28 14:20:03.681886 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.681863    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.685626 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.685600    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 14:20:03.686956 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.686937    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 14:20:03.689063 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.687580    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 14:20:03.689063 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.687862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-whd2d\""
Jul 28 14:20:03.689063 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.688072    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 14:20:03.689063 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.688297    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 14:20:03.696841 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.696820    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6587ff64d9-m9s66"]
Jul 28 14:20:03.804885 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.804589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-oauth-serving-cert\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.804885 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.804653    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-oauth-config\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.804885 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.804677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-service-ca\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.804885 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.804749    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-config\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.804885 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.804779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfgmg\" (UniqueName: \"kubernetes.io/projected/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-kube-api-access-zfgmg\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.804885 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.804805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-serving-cert\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.906512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.906299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-oauth-serving-cert\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.906512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.906349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-oauth-config\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.906512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.906374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-service-ca\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.906512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.906447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-config\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.906512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.906475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfgmg\" (UniqueName: \"kubernetes.io/projected/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-kube-api-access-zfgmg\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.906512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.906502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-serving-cert\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.907715 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.907652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-config\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.907715 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.907651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-oauth-serving-cert\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.907933 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.907907    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-service-ca\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.914305 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.914281    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-serving-cert\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.914912 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.914822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-oauth-config\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.919808 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.919784    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfgmg\" (UniqueName: \"kubernetes.io/projected/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-kube-api-access-zfgmg\") pod \"console-6587ff64d9-m9s66\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") " pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:03.998133 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:03.998102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:04.195443 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:04.195370    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6587ff64d9-m9s66"]
Jul 28 14:20:04.199651 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:04.199622    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a3a80a_07fe_4fa0_b384_1c2a964c626e.slice/crio-d248ef8a31d16f415a9f2c9406051b5162eb77e89ec42c2bf283fb7609232643 WatchSource:0}: Error finding container d248ef8a31d16f415a9f2c9406051b5162eb77e89ec42c2bf283fb7609232643: Status 404 returned error can't find the container with id d248ef8a31d16f415a9f2c9406051b5162eb77e89ec42c2bf283fb7609232643
Jul 28 14:20:04.668641 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:04.668174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6587ff64d9-m9s66" event={"ID":"a7a3a80a-07fe-4fa0-b384-1c2a964c626e","Type":"ContainerStarted","Data":"d248ef8a31d16f415a9f2c9406051b5162eb77e89ec42c2bf283fb7609232643"}
Jul 28 14:20:11.506745 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.506707    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"]
Jul 28 14:20:11.511631 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.510257    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.515408 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.514489    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 14:20:11.515408 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.514636    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lnmkj\""
Jul 28 14:20:11.515408 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.514715    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 14:20:11.515408 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.514934    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 14:20:11.515408 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.514962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 14:20:11.518780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.518633    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 14:20:11.526599 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.526581    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"]
Jul 28 14:20:11.538431 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.538410    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hbt98"]
Jul 28 14:20:11.541856 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.541836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.545280 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.545237    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 14:20:11.546304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.545967    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k2gn6\""
Jul 28 14:20:11.546304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.546164    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 14:20:11.546304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.546192    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.661842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f755e95-904a-44c7-9409-2d05918e0360-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.661888    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-sys\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.661915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-root\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.661944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-wtmp\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.661977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce878207-39d8-47f3-9f53-fc06aa79bc68-metrics-client-ca\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.662001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxc8h\" (UniqueName: \"kubernetes.io/projected/9f755e95-904a-44c7-9409-2d05918e0360-kube-api-access-sxc8h\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.662027    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-accelerators-collector-config\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.662063    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f755e95-904a-44c7-9409-2d05918e0360-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.662107    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f755e95-904a-44c7-9409-2d05918e0360-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.662134    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-textfile\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.662160    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-tls\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.662454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.662200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.663333 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.663201    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4swk\" (UniqueName: \"kubernetes.io/projected/ce878207-39d8-47f3-9f53-fc06aa79bc68-kube-api-access-v4swk\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.707619 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.707111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r" event={"ID":"61937a1d-a055-4c37-bf04-925fce5591f6","Type":"ContainerStarted","Data":"67817b754f36049474b4f710ecf785953ec7c83383dca2840ef4e47cad60b879"}
Jul 28 14:20:11.709438 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.709412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8sngc" event={"ID":"307891dc-8150-4d47-8fa3-849b4b25c3dc","Type":"ContainerStarted","Data":"c438e3019af5f76354aef934f6b27ef9f3fdfbed7f181d2305212cb4bb094fec"}
Jul 28 14:20:11.711951 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.711908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6587ff64d9-m9s66" event={"ID":"a7a3a80a-07fe-4fa0-b384-1c2a964c626e","Type":"ContainerStarted","Data":"abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520"}
Jul 28 14:20:11.728228 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.728164    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7df746cf9d-skw8r" podStartSLOduration=3.8404615250000003 podStartE2EDuration="12.728147389s" totalImagesPullingTime="8.887685864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:11.726169751 +0000 UTC m=+82.014030695" watchObservedRunningTime="2026-07-28 14:20:11.728147389 +0000 UTC m=+82.016008334"
Jul 28 14:20:11.749163 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.748909    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6587ff64d9-m9s66" podStartSLOduration=2.330704064 podStartE2EDuration="8.748895335s" totalImagesPullingTime="6.418191271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:11.747674148 +0000 UTC m=+82.035535116" watchObservedRunningTime="2026-07-28 14:20:11.748895335 +0000 UTC m=+82.036756279"
Jul 28 14:20:11.764399 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-sys\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764399 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-root\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764399 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-sys\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764399 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-wtmp\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764405    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce878207-39d8-47f3-9f53-fc06aa79bc68-metrics-client-ca\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxc8h\" (UniqueName: \"kubernetes.io/projected/9f755e95-904a-44c7-9409-2d05918e0360-kube-api-access-sxc8h\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-accelerators-collector-config\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764498    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f755e95-904a-44c7-9409-2d05918e0360-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-wtmp\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f755e95-904a-44c7-9409-2d05918e0360-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-textfile\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce878207-39d8-47f3-9f53-fc06aa79bc68-root\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-tls\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.764707 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4swk\" (UniqueName: \"kubernetes.io/projected/ce878207-39d8-47f3-9f53-fc06aa79bc68-kube-api-access-v4swk\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.765203 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.764713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f755e95-904a-44c7-9409-2d05918e0360-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.765798 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.765370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-accelerators-collector-config\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.765798 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.765738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce878207-39d8-47f3-9f53-fc06aa79bc68-metrics-client-ca\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.765798 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.765741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-textfile\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.766004 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:20:11.765812    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 14:20:11.766004 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:20:11.765866    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-tls podName:ce878207-39d8-47f3-9f53-fc06aa79bc68 nodeName:}" failed. No retries permitted until 2026-07-28 14:20:12.265849481 +0000 UTC m=+82.553710414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-tls") pod "node-exporter-hbt98" (UID: "ce878207-39d8-47f3-9f53-fc06aa79bc68") : secret "node-exporter-tls" not found
Jul 28 14:20:11.766484 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.766416    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f755e95-904a-44c7-9409-2d05918e0360-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.768551 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.768513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.768946 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.768905    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f755e95-904a-44c7-9409-2d05918e0360-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.770933 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.770890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f755e95-904a-44c7-9409-2d05918e0360-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.777446 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.777405    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxc8h\" (UniqueName: \"kubernetes.io/projected/9f755e95-904a-44c7-9409-2d05918e0360-kube-api-access-sxc8h\") pod \"openshift-state-metrics-b8f49f5c6-8vfnl\" (UID: \"9f755e95-904a-44c7-9409-2d05918e0360\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.778003 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.777959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4swk\" (UniqueName: \"kubernetes.io/projected/ce878207-39d8-47f3-9f53-fc06aa79bc68-kube-api-access-v4swk\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:11.823730 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.823708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"
Jul 28 14:20:11.978792 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:11.978763    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl"]
Jul 28 14:20:11.981023 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:11.980987    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f755e95_904a_44c7_9409_2d05918e0360.slice/crio-e50b42ff4aa277f85a9c865f7c0eb1833cbae107b6083147d487064fa946dec3 WatchSource:0}: Error finding container e50b42ff4aa277f85a9c865f7c0eb1833cbae107b6083147d487064fa946dec3: Status 404 returned error can't find the container with id e50b42ff4aa277f85a9c865f7c0eb1833cbae107b6083147d487064fa946dec3
Jul 28 14:20:12.269890 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.269417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-tls\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:12.275451 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.275402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce878207-39d8-47f3-9f53-fc06aa79bc68-node-exporter-tls\") pod \"node-exporter-hbt98\" (UID: \"ce878207-39d8-47f3-9f53-fc06aa79bc68\") " pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:12.452904 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.452869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hbt98"
Jul 28 14:20:12.471298 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:12.471260    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce878207_39d8_47f3_9f53_fc06aa79bc68.slice/crio-bbaa3774cbf558e2cb710e20c41b40901d8b465486a4f7cb72dfbce026634b9a WatchSource:0}: Error finding container bbaa3774cbf558e2cb710e20c41b40901d8b465486a4f7cb72dfbce026634b9a: Status 404 returned error can't find the container with id bbaa3774cbf558e2cb710e20c41b40901d8b465486a4f7cb72dfbce026634b9a
Jul 28 14:20:12.578287 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.577805    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:20:12.590851 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.589198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.593260 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.593101    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 14:20:12.593743 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.593601    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 14:20:12.594172 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.594156    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 14:20:12.602757 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.596897    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 14:20:12.603011 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.596955    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 14:20:12.603011 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.603013    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:20:12.603189 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.597174    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 14:20:12.603189 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.597135    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 14:20:12.603189 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.597282    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 14:20:12.603433 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.601983    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4tz9t\""
Jul 28 14:20:12.603433 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.597224    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 14:20:12.675493 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-out\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675565    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-web-config\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwzj9\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-kube-api-access-rwzj9\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.675914 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.675854    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.724162 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.724101    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbt98" event={"ID":"ce878207-39d8-47f3-9f53-fc06aa79bc68","Type":"ContainerStarted","Data":"bbaa3774cbf558e2cb710e20c41b40901d8b465486a4f7cb72dfbce026634b9a"}
Jul 28 14:20:12.726803 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.726332    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl" event={"ID":"9f755e95-904a-44c7-9409-2d05918e0360","Type":"ContainerStarted","Data":"39940e87d8f94a28530f1534e38faeee675bddb7e6253909b0a2051da13a4f21"}
Jul 28 14:20:12.726803 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.726389    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl" event={"ID":"9f755e95-904a-44c7-9409-2d05918e0360","Type":"ContainerStarted","Data":"480d1f500c22f1c6f5fba2d5403a4a6345bc592a708acfa03f6e6c634057ad2d"}
Jul 28 14:20:12.726803 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.726419    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl" event={"ID":"9f755e95-904a-44c7-9409-2d05918e0360","Type":"ContainerStarted","Data":"e50b42ff4aa277f85a9c865f7c0eb1833cbae107b6083147d487064fa946dec3"}
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777347    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777433    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-web-config\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwzj9\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-kube-api-access-rwzj9\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-out\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.777729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:20:12.777847    2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 28 14:20:12.778156 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:20:12.777908    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls podName:aa65ecee-681e-462d-b16e-54f88fe03c3f nodeName:}" failed. No retries permitted until 2026-07-28 14:20:13.277887651 +0000 UTC m=+83.565748572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f") : secret "alertmanager-main-tls" not found
Jul 28 14:20:12.779010 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.778293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.779672 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.779645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.785333 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.785287    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.785587 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.785549    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.790691 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.790668    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-web-config\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.793081 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.793056    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-out\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.793376 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.793331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-volume\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.794577 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.794159    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.795423 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.795386    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.795517 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.795408    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.796307 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.796265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwzj9\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-kube-api-access-rwzj9\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:12.811275 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:12.811235    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:13.282789 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:13.282501    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:13.285855 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:13.285799    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:13.502784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:13.502750    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:20:13.670099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:13.670032    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hxz4r"
Jul 28 14:20:13.999150 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:13.999120    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:13.999329 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:13.999177    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:14.000742 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.000712    2579 patch_prober.go:28] interesting pod/console-6587ff64d9-m9s66 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body=
Jul 28 14:20:14.000864 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.000774    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6587ff64d9-m9s66" podUID="a7a3a80a-07fe-4fa0-b384-1c2a964c626e" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused"
Jul 28 14:20:14.498923 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.498887    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"]
Jul 28 14:20:14.518055 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.518024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"]
Jul 28 14:20:14.518242 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.518111    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.521838 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.521512    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 14:20:14.521838 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.521525    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 14:20:14.521838 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.521541    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 14:20:14.521838 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.521654    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fgt8blov8lo2h\""
Jul 28 14:20:14.521838 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.521677    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 14:20:14.521838 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.521669    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 14:20:14.522197 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.521945    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bl2qb\""
Jul 28 14:20:14.592777 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592747    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.592920 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzkj7\" (UniqueName: \"kubernetes.io/projected/be9e4d99-4053-4482-b180-598af0463f71-kube-api-access-nzkj7\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.592920 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.592920 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.592920 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be9e4d99-4053-4482-b180-598af0463f71-metrics-client-ca\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.593055 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592925    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-tls\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.593055 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-grpc-tls\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.593055 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.592998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.693555 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.693523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.693566    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be9e4d99-4053-4482-b180-598af0463f71-metrics-client-ca\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.693709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-tls\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.693763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-grpc-tls\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.693808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.694044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.694095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzkj7\" (UniqueName: \"kubernetes.io/projected/be9e4d99-4053-4482-b180-598af0463f71-kube-api-access-nzkj7\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.694137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.694541 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.694320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be9e4d99-4053-4482-b180-598af0463f71-metrics-client-ca\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.697166 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.697091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.697478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.697451    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.697583 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.697536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-grpc-tls\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.697667 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.697612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.698064 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.698030    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-tls\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.698162 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.698147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be9e4d99-4053-4482-b180-598af0463f71-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.703640 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.703620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzkj7\" (UniqueName: \"kubernetes.io/projected/be9e4d99-4053-4482-b180-598af0463f71-kube-api-access-nzkj7\") pod \"thanos-querier-6bb8897c9b-bd2h7\" (UID: \"be9e4d99-4053-4482-b180-598af0463f71\") " pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:14.830157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:14.830066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:16.015526 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.015491    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5559d6958b-bp8cq"]
Jul 28 14:20:16.021342 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.020588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.023897 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.023871    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ajvvoh47hrjr4\""
Jul 28 14:20:16.025369 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.025153    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 14:20:16.025369 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.025192    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 14:20:16.025369 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.025259    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 14:20:16.025369 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.025333    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 14:20:16.025595 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.025558    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8xsnl\""
Jul 28 14:20:16.033249 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.031882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5559d6958b-bp8cq"]
Jul 28 14:20:16.109006 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.108975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpp8f\" (UniqueName: \"kubernetes.io/projected/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-kube-api-access-lpp8f\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.109006 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.109009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.109243 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.109036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-secret-metrics-server-client-certs\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.109243 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.109064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-client-ca-bundle\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.109243 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.109117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-audit-log\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.109243 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.109140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-secret-metrics-server-tls\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.109243 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.109187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-metrics-server-audit-profiles\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.209770 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.209732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-metrics-server-audit-profiles\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.209956 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.209787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpp8f\" (UniqueName: \"kubernetes.io/projected/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-kube-api-access-lpp8f\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.209956 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.209913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.210056 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.209964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-secret-metrics-server-client-certs\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.210056 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.209997    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-client-ca-bundle\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.210056 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.210045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-audit-log\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.210229 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.210170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-secret-metrics-server-tls\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.210530 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.210498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-audit-log\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.210645 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.210583    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.210879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.210856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-metrics-server-audit-profiles\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.213192 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.213047    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-secret-metrics-server-client-certs\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.213192 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.213047    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-client-ca-bundle\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.213192 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.213143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-secret-metrics-server-tls\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.226920 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.226895    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpp8f\" (UniqueName: \"kubernetes.io/projected/09d54cc9-7641-4acd-b3e0-d33dfb12d2c6-kube-api-access-lpp8f\") pod \"metrics-server-5559d6958b-bp8cq\" (UID: \"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6\") " pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.333512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.333387    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:16.487873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.487840    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6587ff64d9-m9s66"]
Jul 28 14:20:16.535533 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.535500    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55c79d4bc7-dtgst"]
Jul 28 14:20:16.538867 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.538844    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.550151 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.550126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c79d4bc7-dtgst"]
Jul 28 14:20:16.551163 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.551144    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 14:20:16.613760 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.613675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpr92\" (UniqueName: \"kubernetes.io/projected/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-kube-api-access-dpr92\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.613760 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.613757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-serving-cert\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.613969 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.613832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-oauth-serving-cert\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.613969 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.613913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-service-ca\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.614073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.613965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-oauth-config\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.614073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.613992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-config\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.614073 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.614014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-trusted-ca-bundle\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.714753 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.714713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpr92\" (UniqueName: \"kubernetes.io/projected/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-kube-api-access-dpr92\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.714954 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.714783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-serving-cert\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.714954 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.714809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-oauth-serving-cert\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.714954 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.714848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-service-ca\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.714954 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.714895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-oauth-config\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.714954 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.714916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-config\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.714954 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.714934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-trusted-ca-bundle\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.715783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.715724    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-oauth-serving-cert\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.715958 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.715924    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-trusted-ca-bundle\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.715958 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.715941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-config\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.716115 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.715946    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-service-ca\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.717590 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.717570    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-oauth-config\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.717845 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.717815    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-serving-cert\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.724883 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.724858    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpr92\" (UniqueName: \"kubernetes.io/projected/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-kube-api-access-dpr92\") pod \"console-55c79d4bc7-dtgst\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") " pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.786843 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.786810    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-674d9c4964-c68d7"]
Jul 28 14:20:16.793131 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.793104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.796676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.796598    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 14:20:16.796802 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.796704    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 14:20:16.796802 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.796714    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 14:20:16.796926 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.796829    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6p4st\""
Jul 28 14:20:16.797154 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.797134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 14:20:16.799194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.799160    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 14:20:16.803432 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.803413    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 14:20:16.815884 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.815862    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-674d9c4964-c68d7"]
Jul 28 14:20:16.850572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.850536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:16.917381 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917295    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.917381 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.917381 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-telemeter-client-tls\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.917633 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-metrics-client-ca\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.917633 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-serving-certs-ca-bundle\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.917633 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4cwr\" (UniqueName: \"kubernetes.io/projected/265fe75e-edbd-414d-81cd-c08949d3627c-kube-api-access-c4cwr\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.917633 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917536    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-secret-telemeter-client\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:16.917633 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:16.917561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-federate-client-tls\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019079 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019079 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-telemeter-client-tls\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-metrics-client-ca\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-serving-certs-ca-bundle\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019201    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4cwr\" (UniqueName: \"kubernetes.io/projected/265fe75e-edbd-414d-81cd-c08949d3627c-kube-api-access-c4cwr\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-secret-telemeter-client\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.019678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-federate-client-tls\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.020075 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.019993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-serving-certs-ca-bundle\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.020184 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.020130    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.020301 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.020277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265fe75e-edbd-414d-81cd-c08949d3627c-metrics-client-ca\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.022072 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.022050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-telemeter-client-tls\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.022178 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.022156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.022301 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.022285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-federate-client-tls\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.022401 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.022379    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/265fe75e-edbd-414d-81cd-c08949d3627c-secret-telemeter-client\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.034267 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.034240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4cwr\" (UniqueName: \"kubernetes.io/projected/265fe75e-edbd-414d-81cd-c08949d3627c-kube-api-access-c4cwr\") pod \"telemeter-client-674d9c4964-c68d7\" (UID: \"265fe75e-edbd-414d-81cd-c08949d3627c\") " pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.105658 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.105615    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7"
Jul 28 14:20:17.772129 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.772103    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:20:17.783205 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.783182    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.788318 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.788288    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 14:20:17.788778 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.788757    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 14:20:17.788778 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.788773    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 14:20:17.788941 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.788784    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 14:20:17.788941 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.788816    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 14:20:17.788941 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.788835    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e5p576j4h0l3o\""
Jul 28 14:20:17.788941 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.788895    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 14:20:17.790993 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.790729    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 14:20:17.790993 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.790806    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 14:20:17.790993 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.790985    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 14:20:17.791235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.791202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 14:20:17.791719 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.791697    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 14:20:17.791883 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.791865    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 14:20:17.792320 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.792300    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 14:20:17.794621 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.794602    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ws2b2\""
Jul 28 14:20:17.796868 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.796849    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 14:20:17.801491 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.801472    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:20:17.927590 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927765 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927765 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-config-out\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927765 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927765 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-web-config\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927809    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.927939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928082 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928082 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.927985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928082 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928002    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928082 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928178    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-config\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwp9s\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-kube-api-access-fwp9s\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928504 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:17.928504 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:17.928337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.028989 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.028900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-config-out\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.028989 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.028945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.028989 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.028983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-web-config\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029077    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029268    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-config\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwp9s\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-kube-api-access-fwp9s\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.029538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.030405 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.029840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.030405 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.030161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.030405 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.030281    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.030744 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.030568    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.031190 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.031128    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.032366 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.031880    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.032802 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.032777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.033435 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.033411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-web-config\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.033753 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.033730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.034301 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.034279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.035061 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.035015    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.035061 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.035018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-config-out\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.035195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.035137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.035781 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.035737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.036118 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.036093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.036456 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.036419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.036986 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.036948    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.037311 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.037287    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.037758 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.037741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-config\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.046333 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.046304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwp9s\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-kube-api-access-fwp9s\") pod \"prometheus-k8s-0\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:18.094349 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:18.094323    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:23.316014 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.315740    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:20:23.321091 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:23.321051    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa65ecee_681e_462d_b16e_54f88fe03c3f.slice/crio-d98b7d705cd48ab5e894d19f05f18bfd2ead00b25c186daa6126f1db188ad10a WatchSource:0}: Error finding container d98b7d705cd48ab5e894d19f05f18bfd2ead00b25c186daa6126f1db188ad10a: Status 404 returned error can't find the container with id d98b7d705cd48ab5e894d19f05f18bfd2ead00b25c186daa6126f1db188ad10a
Jul 28 14:20:23.541435 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.541408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:20:23.546773 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:23.546741    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8a55bb_e9da_40d1_90a4_c3c968469c10.slice/crio-aa60008e6f702927e474c693ff210ca8a70a96cebd23a413166ed2cf2761a26e WatchSource:0}: Error finding container aa60008e6f702927e474c693ff210ca8a70a96cebd23a413166ed2cf2761a26e: Status 404 returned error can't find the container with id aa60008e6f702927e474c693ff210ca8a70a96cebd23a413166ed2cf2761a26e
Jul 28 14:20:23.770611 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.770552    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c79d4bc7-dtgst"]
Jul 28 14:20:23.776572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.773820    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"]
Jul 28 14:20:23.776572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.775549    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5559d6958b-bp8cq"]
Jul 28 14:20:23.779461 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.777375    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-674d9c4964-c68d7"]
Jul 28 14:20:23.781248 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.780994    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"d98b7d705cd48ab5e894d19f05f18bfd2ead00b25c186daa6126f1db188ad10a"}
Jul 28 14:20:23.782987 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:23.782960    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265fe75e_edbd_414d_81cd_c08949d3627c.slice/crio-aa5c807d49a26fb3d9870b99516e91e83a834109b549a953abc95f73553edda9 WatchSource:0}: Error finding container aa5c807d49a26fb3d9870b99516e91e83a834109b549a953abc95f73553edda9: Status 404 returned error can't find the container with id aa5c807d49a26fb3d9870b99516e91e83a834109b549a953abc95f73553edda9
Jul 28 14:20:23.784069 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.784042    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-brnr7" event={"ID":"beb56aef-8622-474e-85b4-7a7d265fca68","Type":"ContainerStarted","Data":"01f0484a8ad744430d514834840b46c36cb3184de787c922ae60343a31a4eb15"}
Jul 28 14:20:23.785609 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:20:23.785574    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe9e4d99_4053_4482_b180_598af0463f71.slice/crio-d43dca0ef69145a5d7e91ae9775e9b84c3bbcf30af576afe76e2a192e5f8ffa2 WatchSource:0}: Error finding container d43dca0ef69145a5d7e91ae9775e9b84c3bbcf30af576afe76e2a192e5f8ffa2: Status 404 returned error can't find the container with id d43dca0ef69145a5d7e91ae9775e9b84c3bbcf30af576afe76e2a192e5f8ffa2
Jul 28 14:20:23.786733 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.786622    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8sngc" event={"ID":"307891dc-8150-4d47-8fa3-849b4b25c3dc","Type":"ContainerStarted","Data":"71c6f812b353b5dfdb748cd2238db15fa1d359f6d81530811c9a1c9e3498ad6e"}
Jul 28 14:20:23.788015 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.787992    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-brnr7"
Jul 28 14:20:23.789700 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.789486    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce878207-39d8-47f3-9f53-fc06aa79bc68" containerID="ed5d17761dafa652e0e801c55e1727cf8e21d787180335d446bc31c0122c988c" exitCode=0
Jul 28 14:20:23.790143 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.790084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbt98" event={"ID":"ce878207-39d8-47f3-9f53-fc06aa79bc68","Type":"ContainerDied","Data":"ed5d17761dafa652e0e801c55e1727cf8e21d787180335d446bc31c0122c988c"}
Jul 28 14:20:23.793637 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.793611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl" event={"ID":"9f755e95-904a-44c7-9409-2d05918e0360","Type":"ContainerStarted","Data":"5f2ffc3a34f129e317272b8726b0c3cae6875b738d8e71b343816965602f2126"}
Jul 28 14:20:23.795019 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.794999    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-brnr7"
Jul 28 14:20:23.795285 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.795259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"aa60008e6f702927e474c693ff210ca8a70a96cebd23a413166ed2cf2761a26e"}
Jul 28 14:20:23.809256 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:23.809160    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-brnr7" podStartSLOduration=3.220981848 podStartE2EDuration="24.809142251s" totalImagesPullingTime="21.588160403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:23.807864122 +0000 UTC m=+94.095725103" watchObservedRunningTime="2026-07-28 14:20:23.809142251 +0000 UTC m=+94.097003199"
Jul 28 14:20:24.807449 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.807336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq" event={"ID":"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6","Type":"ContainerStarted","Data":"ec30c7bbc19137afe3bd2efac93231ab66b737d3fe46ae045c8af7ad6f26e7d3"}
Jul 28 14:20:24.813396 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.812509    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbt98" event={"ID":"ce878207-39d8-47f3-9f53-fc06aa79bc68","Type":"ContainerStarted","Data":"f82631a4459417e0d6b1b02082ed0428f22d7c61bbac57ae18bbb21d9bec7a8b"}
Jul 28 14:20:24.813396 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.812565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbt98" event={"ID":"ce878207-39d8-47f3-9f53-fc06aa79bc68","Type":"ContainerStarted","Data":"418b9129696b3ebaa2cc779b776db80f498d6f2a4dbad1dc2880ab159d6974d5"}
Jul 28 14:20:24.816139 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.815613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c79d4bc7-dtgst" event={"ID":"31d68335-ae6e-4508-a3ee-1e9aeec2d66f","Type":"ContainerStarted","Data":"6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c"}
Jul 28 14:20:24.816139 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.815651    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c79d4bc7-dtgst" event={"ID":"31d68335-ae6e-4508-a3ee-1e9aeec2d66f","Type":"ContainerStarted","Data":"2de9e844d76be5ee81f84e749d37820fc2ad6e41829b898f643c432d2f25a688"}
Jul 28 14:20:24.817760 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.817701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7" event={"ID":"265fe75e-edbd-414d-81cd-c08949d3627c","Type":"ContainerStarted","Data":"aa5c807d49a26fb3d9870b99516e91e83a834109b549a953abc95f73553edda9"}
Jul 28 14:20:24.820238 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.820121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" event={"ID":"be9e4d99-4053-4482-b180-598af0463f71","Type":"ContainerStarted","Data":"d43dca0ef69145a5d7e91ae9775e9b84c3bbcf30af576afe76e2a192e5f8ffa2"}
Jul 28 14:20:24.842094 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.842030    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hbt98" podStartSLOduration=3.084359627 podStartE2EDuration="13.842011683s" totalImagesPullingTime="10.674216004s" totalInitContainerRuntime="83.436052ms" isStatefulPod=true podCreationTimestamp="2026-07-28 14:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:24.840176981 +0000 UTC m=+95.128037925" watchObservedRunningTime="2026-07-28 14:20:24.842011683 +0000 UTC m=+95.129872627"
Jul 28 14:20:24.842554 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.842513    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-8vfnl" podStartSLOduration=2.8181420409999998 podStartE2EDuration="13.842498117s" totalImagesPullingTime="11.024356076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:23.884500331 +0000 UTC m=+94.172361280" watchObservedRunningTime="2026-07-28 14:20:24.842498117 +0000 UTC m=+95.130359062"
Jul 28 14:20:24.869137 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:24.868722    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55c79d4bc7-dtgst" podStartSLOduration=8.868705756 podStartE2EDuration="8.868705756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:24.867353718 +0000 UTC m=+95.155214654" watchObservedRunningTime="2026-07-28 14:20:24.868705756 +0000 UTC m=+95.156566682"
Jul 28 14:20:26.848637 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:26.848594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490"}
Jul 28 14:20:26.851055 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:26.850951    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:26.851313 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:26.851128    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:26.852335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:26.852165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"}
Jul 28 14:20:26.857472 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:26.857411    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:27.861252 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:27.860177    2579 generic.go:356] "Generic (PLEG): container finished" podID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222" exitCode=0
Jul 28 14:20:27.861252 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:27.860370    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"}
Jul 28 14:20:27.864304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:27.864279    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerID="a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490" exitCode=0
Jul 28 14:20:27.864417 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:27.864353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490"}
Jul 28 14:20:27.869630 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:27.869607    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:20:31.895236 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.894836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7" event={"ID":"265fe75e-edbd-414d-81cd-c08949d3627c","Type":"ContainerStarted","Data":"35e1b6d8b0f405a531a5ef0b48377687c6eb910f570cb89141647bebf4fb4937"}
Jul 28 14:20:31.895236 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.894902    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7" event={"ID":"265fe75e-edbd-414d-81cd-c08949d3627c","Type":"ContainerStarted","Data":"6e2fdc045db0b0c5eead084a91d76c47790e070a4718f210dbcd6c89badfc3ff"}
Jul 28 14:20:31.895236 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.894933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7" event={"ID":"265fe75e-edbd-414d-81cd-c08949d3627c","Type":"ContainerStarted","Data":"6f5ac35a977d4ca2d9319729d18fd43eecbd7cc4d8b5ef545486f113dfa34648"}
Jul 28 14:20:31.900904 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.900876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" event={"ID":"be9e4d99-4053-4482-b180-598af0463f71","Type":"ContainerStarted","Data":"be03f1150637671c3952b1af1ee965b1f3f0551879f935ba99b052caf4ac14e2"}
Jul 28 14:20:31.901022 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.900945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" event={"ID":"be9e4d99-4053-4482-b180-598af0463f71","Type":"ContainerStarted","Data":"0c79b50203ca918963b14d55f7544949853f9483d59f3439ef0f7fb4f4a974f2"}
Jul 28 14:20:31.901022 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.900988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" event={"ID":"be9e4d99-4053-4482-b180-598af0463f71","Type":"ContainerStarted","Data":"21549a14775b145ff32cc1053a94097029fe8b3e55f805885245b16e1eb14ec7"}
Jul 28 14:20:31.903851 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.903795    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq" event={"ID":"09d54cc9-7641-4acd-b3e0-d33dfb12d2c6","Type":"ContainerStarted","Data":"f7ddbdca6bd215900691cfcd998a134483f9bedb024bce779fd380f6e98a9865"}
Jul 28 14:20:31.907153 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.907112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8sngc" event={"ID":"307891dc-8150-4d47-8fa3-849b4b25c3dc","Type":"ContainerStarted","Data":"4bb61cc19b229e9e2d4b90267a5b701bf073fc6ea040cc465686fed844fb9a2f"}
Jul 28 14:20:31.943138 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:31.943077    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-674d9c4964-c68d7" podStartSLOduration=8.373960878 podStartE2EDuration="15.943053595s" totalImagesPullingTime="7.569092717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:31.937881372 +0000 UTC m=+102.225742315" watchObservedRunningTime="2026-07-28 14:20:31.943053595 +0000 UTC m=+102.230914539"
Jul 28 14:20:32.013236 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:32.013052    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq" podStartSLOduration=9.479968884 podStartE2EDuration="17.01302775s" totalImagesPullingTime="7.533058866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:31.975049238 +0000 UTC m=+102.262910232" watchObservedRunningTime="2026-07-28 14:20:32.01302775 +0000 UTC m=+102.300888691"
Jul 28 14:20:32.018232 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:32.014099    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8sngc" podStartSLOduration=36.561725649 podStartE2EDuration="1m6.014081844s" totalImagesPullingTime="29.452356195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:19:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:32.010681618 +0000 UTC m=+102.298542564" watchObservedRunningTime="2026-07-28 14:20:32.014081844 +0000 UTC m=+102.301942789"
Jul 28 14:20:33.919273 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:33.919236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140"}
Jul 28 14:20:34.670642 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:34.670613    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9wstk"
Jul 28 14:20:35.931334 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.931299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" event={"ID":"be9e4d99-4053-4482-b180-598af0463f71","Type":"ContainerStarted","Data":"dbcd2217fdad30cabc96291b814def470ff8301d63f3d72791a0fd1deddea16f"}
Jul 28 14:20:35.931877 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.931412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" event={"ID":"be9e4d99-4053-4482-b180-598af0463f71","Type":"ContainerStarted","Data":"75382113a87a5154fa1b74b80772ff45520e55a43fe8656c88aabd9134fbaaa5"}
Jul 28 14:20:35.931877 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.931460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" event={"ID":"be9e4d99-4053-4482-b180-598af0463f71","Type":"ContainerStarted","Data":"d354f3b7d30e35e0268a6a40c00a1e65a6fdf9e317b591ccea0be337946213de"}
Jul 28 14:20:35.931877 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.931514    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:35.934558 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.934534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98"}
Jul 28 14:20:35.934679 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.934598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332"}
Jul 28 14:20:35.934679 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.934636    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa"}
Jul 28 14:20:35.934679 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.934674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777"}
Jul 28 14:20:35.934829 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.934700    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerStarted","Data":"bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17"}
Jul 28 14:20:35.937944 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.937920    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"}
Jul 28 14:20:35.938042 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.937991    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"}
Jul 28 14:20:35.938042 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.938030    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"}
Jul 28 14:20:35.938156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.938074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"}
Jul 28 14:20:35.938156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.938115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"}
Jul 28 14:20:35.938156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.938149    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerStarted","Data":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"}
Jul 28 14:20:35.965570 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:35.965530    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7" podStartSLOduration=11.378330649 podStartE2EDuration="21.965515143s" totalImagesPullingTime="10.587184494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:35.962509387 +0000 UTC m=+106.250370356" watchObservedRunningTime="2026-07-28 14:20:35.965515143 +0000 UTC m=+106.253376120"
Jul 28 14:20:36.002849 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:36.002807    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=9.55564675 podStartE2EDuration="19.002793065s" totalImagesPullingTime="9.412011065s" totalInitContainerRuntime="35.13525ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:36.000686559 +0000 UTC m=+106.288547523" watchObservedRunningTime="2026-07-28 14:20:36.002793065 +0000 UTC m=+106.290654008"
Jul 28 14:20:36.040260 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:36.040201    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=16.216195077 podStartE2EDuration="24.040188788s" totalImagesPullingTime="7.766116903s" totalInitContainerRuntime="57.876808ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:20:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:20:36.039551955 +0000 UTC m=+106.327412945" watchObservedRunningTime="2026-07-28 14:20:36.040188788 +0000 UTC m=+106.328049730"
Jul 28 14:20:36.334287 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:36.334189    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:36.334287 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:36.334278    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:36.949237 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:36.949192    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6bb8897c9b-bd2h7"
Jul 28 14:20:38.094726 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:38.094691    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:20:41.507710 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.507648    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6587ff64d9-m9s66" podUID="a7a3a80a-07fe-4fa0-b384-1c2a964c626e" containerName="console" containerID="cri-o://abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520" gracePeriod=15
Jul 28 14:20:41.777173 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.777111    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:41.891346 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891315    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-service-ca\") pod \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") "
Jul 28 14:20:41.891346 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891349    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfgmg\" (UniqueName: \"kubernetes.io/projected/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-kube-api-access-zfgmg\") pod \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") "
Jul 28 14:20:41.891539 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891392    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-config\") pod \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") "
Jul 28 14:20:41.891539 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891444    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-serving-cert\") pod \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") "
Jul 28 14:20:41.891539 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891496    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-oauth-serving-cert\") pod \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") "
Jul 28 14:20:41.891539 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891517    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-oauth-config\") pod \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\" (UID: \"a7a3a80a-07fe-4fa0-b384-1c2a964c626e\") "
Jul 28 14:20:41.891871 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891837    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-config" (OuterVolumeSpecName: "console-config") pod "a7a3a80a-07fe-4fa0-b384-1c2a964c626e" (UID: "a7a3a80a-07fe-4fa0-b384-1c2a964c626e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:20:41.891980 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891847    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-service-ca" (OuterVolumeSpecName: "service-ca") pod "a7a3a80a-07fe-4fa0-b384-1c2a964c626e" (UID: "a7a3a80a-07fe-4fa0-b384-1c2a964c626e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:20:41.891980 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.891880    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a7a3a80a-07fe-4fa0-b384-1c2a964c626e" (UID: "a7a3a80a-07fe-4fa0-b384-1c2a964c626e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:20:41.893731 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.893701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-kube-api-access-zfgmg" (OuterVolumeSpecName: "kube-api-access-zfgmg") pod "a7a3a80a-07fe-4fa0-b384-1c2a964c626e" (UID: "a7a3a80a-07fe-4fa0-b384-1c2a964c626e"). InnerVolumeSpecName "kube-api-access-zfgmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:20:41.893824 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.893731    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a7a3a80a-07fe-4fa0-b384-1c2a964c626e" (UID: "a7a3a80a-07fe-4fa0-b384-1c2a964c626e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:20:41.893824 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.893775    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a7a3a80a-07fe-4fa0-b384-1c2a964c626e" (UID: "a7a3a80a-07fe-4fa0-b384-1c2a964c626e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:20:41.968302 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.968267    2579 generic.go:356] "Generic (PLEG): container finished" podID="a7a3a80a-07fe-4fa0-b384-1c2a964c626e" containerID="abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520" exitCode=0
Jul 28 14:20:41.968473 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.968319    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6587ff64d9-m9s66"
Jul 28 14:20:41.968473 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.968331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6587ff64d9-m9s66" event={"ID":"a7a3a80a-07fe-4fa0-b384-1c2a964c626e","Type":"ContainerDied","Data":"abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520"}
Jul 28 14:20:41.968473 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.968377    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6587ff64d9-m9s66" event={"ID":"a7a3a80a-07fe-4fa0-b384-1c2a964c626e","Type":"ContainerDied","Data":"d248ef8a31d16f415a9f2c9406051b5162eb77e89ec42c2bf283fb7609232643"}
Jul 28 14:20:41.968473 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.968397    2579 scope.go:117] "RemoveContainer" containerID="abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520"
Jul 28 14:20:41.977755 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.977737    2579 scope.go:117] "RemoveContainer" containerID="abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520"
Jul 28 14:20:41.978043 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:20:41.978022    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520\": container with ID starting with abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520 not found: ID does not exist" containerID="abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520"
Jul 28 14:20:41.978098 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.978051    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520"} err="failed to get container status \"abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520\": rpc error: code = NotFound desc = could not find container \"abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520\": container with ID starting with abbb75d697b9891562f2a949aabd1a8f66992ddbd8c4bd63638fd248b9767520 not found: ID does not exist"
Jul 28 14:20:41.991068 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.991043    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6587ff64d9-m9s66"]
Jul 28 14:20:41.992764 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.992747    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:41.992822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.992766    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-serving-cert\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:41.992822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.992777    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-oauth-serving-cert\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:41.992822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.992786    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-console-oauth-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:41.992822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.992795    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-service-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:41.992822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.992803    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfgmg\" (UniqueName: \"kubernetes.io/projected/a7a3a80a-07fe-4fa0-b384-1c2a964c626e-kube-api-access-zfgmg\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:20:41.995612 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:41.995584    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6587ff64d9-m9s66"]
Jul 28 14:20:42.193538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:42.193135    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7a3a80a-07fe-4fa0-b384-1c2a964c626e" path="/var/lib/kubelet/pods/a7a3a80a-07fe-4fa0-b384-1c2a964c626e/volumes"
Jul 28 14:20:44.548303 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:44.548263    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c79d4bc7-dtgst"]
Jul 28 14:20:52.016728 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:52.016693    2579 generic.go:356] "Generic (PLEG): container finished" podID="0e9d31a5-b09b-4416-a824-4c049c13d215" containerID="5c8a755e0d35adfe537155a54b884badbdb8955342393bc3349603f35337b284" exitCode=0
Jul 28 14:20:52.017158 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:52.016761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm" event={"ID":"0e9d31a5-b09b-4416-a824-4c049c13d215","Type":"ContainerDied","Data":"5c8a755e0d35adfe537155a54b884badbdb8955342393bc3349603f35337b284"}
Jul 28 14:20:52.017158 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:52.017153    2579 scope.go:117] "RemoveContainer" containerID="5c8a755e0d35adfe537155a54b884badbdb8955342393bc3349603f35337b284"
Jul 28 14:20:53.022496 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:53.022463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7gbjm" event={"ID":"0e9d31a5-b09b-4416-a824-4c049c13d215","Type":"ContainerStarted","Data":"25c85176923559f000cb3af79f101734495feebe407cb4ae6fa6551bf0f72aca"}
Jul 28 14:20:56.338892 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:56.338866    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:20:56.342822 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:20:56.342803    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5559d6958b-bp8cq"
Jul 28 14:21:09.567637 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.567589    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55c79d4bc7-dtgst" podUID="31d68335-ae6e-4508-a3ee-1e9aeec2d66f" containerName="console" containerID="cri-o://6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c" gracePeriod=15
Jul 28 14:21:09.846400 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.846339    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:21:09.931784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.931759    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpr92\" (UniqueName: \"kubernetes.io/projected/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-kube-api-access-dpr92\") pod \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") "
Jul 28 14:21:09.931946 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.931795    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-config\") pod \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") "
Jul 28 14:21:09.931946 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.931821    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-oauth-serving-cert\") pod \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") "
Jul 28 14:21:09.931946 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.931839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-trusted-ca-bundle\") pod \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") "
Jul 28 14:21:09.931946 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.931866    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-service-ca\") pod \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") "
Jul 28 14:21:09.931946 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.931924    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-serving-cert\") pod \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") "
Jul 28 14:21:09.931946 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.931945    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-oauth-config\") pod \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\" (UID: \"31d68335-ae6e-4508-a3ee-1e9aeec2d66f\") "
Jul 28 14:21:09.932283 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.932251    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "31d68335-ae6e-4508-a3ee-1e9aeec2d66f" (UID: "31d68335-ae6e-4508-a3ee-1e9aeec2d66f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:09.933820 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.933794    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-kube-api-access-dpr92" (OuterVolumeSpecName: "kube-api-access-dpr92") pod "31d68335-ae6e-4508-a3ee-1e9aeec2d66f" (UID: "31d68335-ae6e-4508-a3ee-1e9aeec2d66f"). InnerVolumeSpecName "kube-api-access-dpr92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:21:09.933988 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.933974    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "31d68335-ae6e-4508-a3ee-1e9aeec2d66f" (UID: "31d68335-ae6e-4508-a3ee-1e9aeec2d66f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:09.934042 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.934023    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "31d68335-ae6e-4508-a3ee-1e9aeec2d66f" (UID: "31d68335-ae6e-4508-a3ee-1e9aeec2d66f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:09.935291 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.935263    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-config" (OuterVolumeSpecName: "console-config") pod "31d68335-ae6e-4508-a3ee-1e9aeec2d66f" (UID: "31d68335-ae6e-4508-a3ee-1e9aeec2d66f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:09.935367 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.935356    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-service-ca" (OuterVolumeSpecName: "service-ca") pod "31d68335-ae6e-4508-a3ee-1e9aeec2d66f" (UID: "31d68335-ae6e-4508-a3ee-1e9aeec2d66f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:09.935418 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:09.935391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "31d68335-ae6e-4508-a3ee-1e9aeec2d66f" (UID: "31d68335-ae6e-4508-a3ee-1e9aeec2d66f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:10.032524 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.032497    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-serving-cert\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:10.032524 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.032517    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-oauth-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:10.032524 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.032527    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpr92\" (UniqueName: \"kubernetes.io/projected/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-kube-api-access-dpr92\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:10.032701 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.032536    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-console-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:10.032701 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.032544    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-oauth-serving-cert\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:10.032701 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.032552    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-trusted-ca-bundle\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:10.032701 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.032562    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31d68335-ae6e-4508-a3ee-1e9aeec2d66f-service-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:10.092370 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.092335    2579 generic.go:356] "Generic (PLEG): container finished" podID="31d68335-ae6e-4508-a3ee-1e9aeec2d66f" containerID="6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c" exitCode=0
Jul 28 14:21:10.092486 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.092385    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c79d4bc7-dtgst"
Jul 28 14:21:10.092486 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.092426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c79d4bc7-dtgst" event={"ID":"31d68335-ae6e-4508-a3ee-1e9aeec2d66f","Type":"ContainerDied","Data":"6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c"}
Jul 28 14:21:10.092486 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.092479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c79d4bc7-dtgst" event={"ID":"31d68335-ae6e-4508-a3ee-1e9aeec2d66f","Type":"ContainerDied","Data":"2de9e844d76be5ee81f84e749d37820fc2ad6e41829b898f643c432d2f25a688"}
Jul 28 14:21:10.092624 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.092501    2579 scope.go:117] "RemoveContainer" containerID="6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c"
Jul 28 14:21:10.101614 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.101594    2579 scope.go:117] "RemoveContainer" containerID="6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c"
Jul 28 14:21:10.101858 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:10.101836    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c\": container with ID starting with 6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c not found: ID does not exist" containerID="6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c"
Jul 28 14:21:10.101948 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.101861    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c"} err="failed to get container status \"6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c\": rpc error: code = NotFound desc = could not find container \"6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c\": container with ID starting with 6b4dd289fce9f5881769c4d52e6aeccc13012d8d413eb6d75c4d0770903b873c not found: ID does not exist"
Jul 28 14:21:10.116631 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.116613    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c79d4bc7-dtgst"]
Jul 28 14:21:10.118246 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.118228    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55c79d4bc7-dtgst"]
Jul 28 14:21:10.192119 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:10.192084    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d68335-ae6e-4508-a3ee-1e9aeec2d66f" path="/var/lib/kubelet/pods/31d68335-ae6e-4508-a3ee-1e9aeec2d66f/volumes"
Jul 28 14:21:18.094894 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:18.094854    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:18.165758 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:18.165732    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:18.180559 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:18.180533    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:31.930710 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:31.930676    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:21:31.931387 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:31.930977    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="alertmanager" containerID="cri-o://50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140" gracePeriod=120
Jul 28 14:21:31.931387 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:31.931036    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-metric" containerID="cri-o://3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332" gracePeriod=120
Jul 28 14:21:31.931387 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:31.931062    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-web" containerID="cri-o://3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777" gracePeriod=120
Jul 28 14:21:31.931387 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:31.931153    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="prom-label-proxy" containerID="cri-o://abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98" gracePeriod=120
Jul 28 14:21:31.931387 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:31.931110    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="config-reloader" containerID="cri-o://bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17" gracePeriod=120
Jul 28 14:21:31.931387 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:31.931240    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy" containerID="cri-o://db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa" gracePeriod=120
Jul 28 14:21:32.181738 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181658    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerID="abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98" exitCode=0
Jul 28 14:21:32.181738 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181684    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerID="db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa" exitCode=0
Jul 28 14:21:32.181738 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181691    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerID="3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777" exitCode=0
Jul 28 14:21:32.181738 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181698    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerID="bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17" exitCode=0
Jul 28 14:21:32.181738 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181703    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerID="50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140" exitCode=0
Jul 28 14:21:32.182059 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98"}
Jul 28 14:21:32.182059 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa"}
Jul 28 14:21:32.182059 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777"}
Jul 28 14:21:32.182059 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181893    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17"}
Jul 28 14:21:32.182059 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:32.181931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140"}
Jul 28 14:21:33.182087 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.182063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.188372 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.188344    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerID="3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332" exitCode=0
Jul 28 14:21:33.188503 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.188384    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332"}
Jul 28 14:21:33.188503 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.188404    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.188503 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.188462    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aa65ecee-681e-462d-b16e-54f88fe03c3f","Type":"ContainerDied","Data":"d98b7d705cd48ab5e894d19f05f18bfd2ead00b25c186daa6126f1db188ad10a"}
Jul 28 14:21:33.188659 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.188511    2579 scope.go:117] "RemoveContainer" containerID="abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98"
Jul 28 14:21:33.195591 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.195571    2579 scope.go:117] "RemoveContainer" containerID="3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332"
Jul 28 14:21:33.202296 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.202281    2579 scope.go:117] "RemoveContainer" containerID="db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa"
Jul 28 14:21:33.208603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.208588    2579 scope.go:117] "RemoveContainer" containerID="3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777"
Jul 28 14:21:33.216619 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.216563    2579 scope.go:117] "RemoveContainer" containerID="bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17"
Jul 28 14:21:33.225304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.225287    2579 scope.go:117] "RemoveContainer" containerID="50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140"
Jul 28 14:21:33.228587 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.228564    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-main-db\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.228684 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.228618    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-cluster-tls-config\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.228684 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.228675    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-web-config\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.228810 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.228726    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-metrics-client-ca\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.229345 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.229079    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:21:33.229345 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.229171    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:33.234716 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.234693    2579 scope.go:117] "RemoveContainer" containerID="a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490"
Jul 28 14:21:33.235515 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.235492    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:33.243228 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.243190    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-web-config" (OuterVolumeSpecName: "web-config") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:33.250334 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.250316    2579 scope.go:117] "RemoveContainer" containerID="abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98"
Jul 28 14:21:33.250610 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:33.250587    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98\": container with ID starting with abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98 not found: ID does not exist" containerID="abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98"
Jul 28 14:21:33.250676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.250622    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98"} err="failed to get container status \"abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98\": rpc error: code = NotFound desc = could not find container \"abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98\": container with ID starting with abc77f2b9b922f7fcef1236749e65444492a765692517e84e5ffd8d867a0df98 not found: ID does not exist"
Jul 28 14:21:33.250676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.250650    2579 scope.go:117] "RemoveContainer" containerID="3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332"
Jul 28 14:21:33.250916 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:33.250899    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332\": container with ID starting with 3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332 not found: ID does not exist" containerID="3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332"
Jul 28 14:21:33.250968 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.250921    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332"} err="failed to get container status \"3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332\": rpc error: code = NotFound desc = could not find container \"3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332\": container with ID starting with 3f0a277a52c8c8ac58fe46b8cc83d54e24d19df6ed30b61916acd0be51a47332 not found: ID does not exist"
Jul 28 14:21:33.250968 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.250939    2579 scope.go:117] "RemoveContainer" containerID="db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa"
Jul 28 14:21:33.251198 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:33.251180    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa\": container with ID starting with db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa not found: ID does not exist" containerID="db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa"
Jul 28 14:21:33.251291 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.251206    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa"} err="failed to get container status \"db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa\": rpc error: code = NotFound desc = could not find container \"db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa\": container with ID starting with db75b7652df95e7aa8b09884003d8ec6f8411088c1d0356d586fcc00feba79aa not found: ID does not exist"
Jul 28 14:21:33.251291 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.251252    2579 scope.go:117] "RemoveContainer" containerID="3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777"
Jul 28 14:21:33.251508 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:33.251490    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777\": container with ID starting with 3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777 not found: ID does not exist" containerID="3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777"
Jul 28 14:21:33.251570 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.251515    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777"} err="failed to get container status \"3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777\": rpc error: code = NotFound desc = could not find container \"3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777\": container with ID starting with 3749b1bfc274fb7e765c8cdadfc6e6277d9f7dac2ec2fb3791da8025f9e43777 not found: ID does not exist"
Jul 28 14:21:33.251570 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.251533    2579 scope.go:117] "RemoveContainer" containerID="bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17"
Jul 28 14:21:33.251792 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:33.251776    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17\": container with ID starting with bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17 not found: ID does not exist" containerID="bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17"
Jul 28 14:21:33.251842 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.251796    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17"} err="failed to get container status \"bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17\": rpc error: code = NotFound desc = could not find container \"bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17\": container with ID starting with bad3ec695017b9d31f125c74a2150d848c7b2bc618b93007984c4763fbd3dc17 not found: ID does not exist"
Jul 28 14:21:33.251842 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.251810    2579 scope.go:117] "RemoveContainer" containerID="50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140"
Jul 28 14:21:33.252048 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:33.252032    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140\": container with ID starting with 50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140 not found: ID does not exist" containerID="50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140"
Jul 28 14:21:33.252109 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.252054    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140"} err="failed to get container status \"50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140\": rpc error: code = NotFound desc = could not find container \"50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140\": container with ID starting with 50ce875bbcd95dcbe9d37e48654f156a9a4b653c75897b4d3734d4945528e140 not found: ID does not exist"
Jul 28 14:21:33.252109 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.252074    2579 scope.go:117] "RemoveContainer" containerID="a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490"
Jul 28 14:21:33.252342 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:33.252324    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490\": container with ID starting with a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490 not found: ID does not exist" containerID="a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490"
Jul 28 14:21:33.252399 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.252345    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490"} err="failed to get container status \"a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490\": rpc error: code = NotFound desc = could not find container \"a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490\": container with ID starting with a226b3bc1235f19ad66a219892f7b9155902f35c34dc60a425ad0e7425cb6490 not found: ID does not exist"
Jul 28 14:21:33.329839 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.329783    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-trusted-ca-bundle\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.329839 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.329813    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.329852    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwzj9\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-kube-api-access-rwzj9\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.329871    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-tls-assets\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.329916    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.329948    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-volume\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.329964    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330022    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-web\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330419 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330056    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-out\") pod \"aa65ecee-681e-462d-b16e-54f88fe03c3f\" (UID: \"aa65ecee-681e-462d-b16e-54f88fe03c3f\") "
Jul 28 14:21:33.330419 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330240    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:33.330419 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330381    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-cluster-tls-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.330419 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330400    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-web-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.330419 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330414    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.330677 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330427    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa65ecee-681e-462d-b16e-54f88fe03c3f-metrics-client-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.330677 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.330442    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-alertmanager-main-db\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.332455 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.332425    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-volume" (OuterVolumeSpecName: "config-volume") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:33.332714 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.332691    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-out" (OuterVolumeSpecName: "config-out") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:21:33.332784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.332758    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-kube-api-access-rwzj9" (OuterVolumeSpecName: "kube-api-access-rwzj9") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "kube-api-access-rwzj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:21:33.332784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.332768    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:33.332859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.332785    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:33.332859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.332807    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:21:33.332955 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.332936    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:33.333503 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.333486    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "aa65ecee-681e-462d-b16e-54f88fe03c3f" (UID: "aa65ecee-681e-462d-b16e-54f88fe03c3f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:33.430819 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430782    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-volume\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.430819 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430812    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.430819 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430823    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.431004 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430832    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa65ecee-681e-462d-b16e-54f88fe03c3f-config-out\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.431004 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430841    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.431004 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430851    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwzj9\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-kube-api-access-rwzj9\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.431004 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430859    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa65ecee-681e-462d-b16e-54f88fe03c3f-tls-assets\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.431004 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.430868    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aa65ecee-681e-462d-b16e-54f88fe03c3f-secret-alertmanager-main-tls\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:33.514963 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.514935    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:21:33.519882 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.519861    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:21:33.549978 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.549954    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:21:33.550274 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550262    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d68335-ae6e-4508-a3ee-1e9aeec2d66f" containerName="console"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550275    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d68335-ae6e-4508-a3ee-1e9aeec2d66f" containerName="console"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550283    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550289    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550299    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-web"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550305    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-web"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550314    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-metric"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550320    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-metric"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550329    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="prom-label-proxy"
Jul 28 14:21:33.550335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550334    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="prom-label-proxy"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550343    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="init-config-reloader"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550348    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="init-config-reloader"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550354    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="config-reloader"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550359    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="config-reloader"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550370    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a3a80a-07fe-4fa0-b384-1c2a964c626e" containerName="console"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550375    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a3a80a-07fe-4fa0-b384-1c2a964c626e" containerName="console"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550382    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="alertmanager"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550387    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="alertmanager"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550445    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="prom-label-proxy"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550451    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7a3a80a-07fe-4fa0-b384-1c2a964c626e" containerName="console"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550458    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-web"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550463    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d68335-ae6e-4508-a3ee-1e9aeec2d66f" containerName="console"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550468    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy-metric"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550475    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="kube-rbac-proxy"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550483    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="alertmanager"
Jul 28 14:21:33.550671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.550490    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" containerName="config-reloader"
Jul 28 14:21:33.555679 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.555665    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.559340 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.559311    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 14:21:33.559446 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.559385    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 14:21:33.559833 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.559817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 14:21:33.559925 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.559880    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 14:21:33.559925 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.559893    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 14:21:33.560018 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.559925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 14:21:33.560018 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.559929    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 14:21:33.560781 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.560764    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 14:21:33.560881 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.560782    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4tz9t\""
Jul 28 14:21:33.564666 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.564648    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 14:21:33.572307 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.572288    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:21:33.632888 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.632803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jww5t\" (UniqueName: \"kubernetes.io/projected/1192e4e6-7d06-45d0-b703-69d87e0dce78-kube-api-access-jww5t\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.632888 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.632842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1192e4e6-7d06-45d0-b703-69d87e0dce78-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633060 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.632887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1192e4e6-7d06-45d0-b703-69d87e0dce78-config-out\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633060 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.632914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633060 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.632937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1192e4e6-7d06-45d0-b703-69d87e0dce78-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633060 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.632982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1192e4e6-7d06-45d0-b703-69d87e0dce78-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633060 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.633042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-config-volume\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633297 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.633062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1192e4e6-7d06-45d0-b703-69d87e0dce78-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633297 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.633083    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-web-config\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633297 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.633139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633297 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.633167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633297 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.633196    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.633297 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.633234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734661 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1192e4e6-7d06-45d0-b703-69d87e0dce78-config-out\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734661 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1192e4e6-7d06-45d0-b703-69d87e0dce78-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1192e4e6-7d06-45d0-b703-69d87e0dce78-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-config-volume\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1192e4e6-7d06-45d0-b703-69d87e0dce78-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-web-config\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734890    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.734936 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.735352 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.735352 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.734984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jww5t\" (UniqueName: \"kubernetes.io/projected/1192e4e6-7d06-45d0-b703-69d87e0dce78-kube-api-access-jww5t\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.735352 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.735012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1192e4e6-7d06-45d0-b703-69d87e0dce78-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.735352 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.735255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1192e4e6-7d06-45d0-b703-69d87e0dce78-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.736270 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.735852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1192e4e6-7d06-45d0-b703-69d87e0dce78-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.736270 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.736093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1192e4e6-7d06-45d0-b703-69d87e0dce78-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.737572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.737546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1192e4e6-7d06-45d0-b703-69d87e0dce78-config-out\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.738071 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.738047    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.738178 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.738150    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.738269 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.738173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.738577 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.738553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-web-config\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.738745 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.738723    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-config-volume\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.739455 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.739434    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1192e4e6-7d06-45d0-b703-69d87e0dce78-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.739547 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.739501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.739770 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.739753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1192e4e6-7d06-45d0-b703-69d87e0dce78-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.744647 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.744627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jww5t\" (UniqueName: \"kubernetes.io/projected/1192e4e6-7d06-45d0-b703-69d87e0dce78-kube-api-access-jww5t\") pod \"alertmanager-main-0\" (UID: \"1192e4e6-7d06-45d0-b703-69d87e0dce78\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:33.865293 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:33.865263    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 14:21:34.002030 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:34.002008    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 14:21:34.003617 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:21:34.003586    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1192e4e6_7d06_45d0_b703_69d87e0dce78.slice/crio-5450a7f10078012d1e24cf9655798437770d4317cca4585337dbe373739ebe18 WatchSource:0}: Error finding container 5450a7f10078012d1e24cf9655798437770d4317cca4585337dbe373739ebe18: Status 404 returned error can't find the container with id 5450a7f10078012d1e24cf9655798437770d4317cca4585337dbe373739ebe18
Jul 28 14:21:34.192762 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:34.192733    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa65ecee-681e-462d-b16e-54f88fe03c3f" path="/var/lib/kubelet/pods/aa65ecee-681e-462d-b16e-54f88fe03c3f/volumes"
Jul 28 14:21:34.194474 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:34.194452    2579 generic.go:356] "Generic (PLEG): container finished" podID="1192e4e6-7d06-45d0-b703-69d87e0dce78" containerID="c89447834551b44a111a1a4982aea2edb3752a54866c95c0a5d87db02b271af7" exitCode=0
Jul 28 14:21:34.194674 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:34.194505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerDied","Data":"c89447834551b44a111a1a4982aea2edb3752a54866c95c0a5d87db02b271af7"}
Jul 28 14:21:34.194674 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:34.194552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerStarted","Data":"5450a7f10078012d1e24cf9655798437770d4317cca4585337dbe373739ebe18"}
Jul 28 14:21:35.202410 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:35.202371    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerStarted","Data":"40fca9b8c3dc4a21888f6b7c44f94b811d678131645a61f9bf44847acf9db8de"}
Jul 28 14:21:35.202828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:35.202431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerStarted","Data":"f8d40eec5767c2a4384bb4562347819a09b488693fc8720cbaa9f66151453a38"}
Jul 28 14:21:35.202828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:35.202457    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerStarted","Data":"410ff4df91d41dbc6ac2ddcb75137a09e736798e8e5594cf6636644edc02c136"}
Jul 28 14:21:35.202828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:35.202481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerStarted","Data":"a2a6a663a9433683987cc0cf259a6af65fd552b61a01f815514e0849219f7b1a"}
Jul 28 14:21:35.202828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:35.202505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerStarted","Data":"82c1de51d8299aa4c48114ea0f618cabb89c945a049fc58661479811bbd84df9"}
Jul 28 14:21:35.202828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:35.202528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1192e4e6-7d06-45d0-b703-69d87e0dce78","Type":"ContainerStarted","Data":"738063cad5b1aa27dea40d2ee34bd9f355d322b120c14c58362270b7792c7643"}
Jul 28 14:21:35.240375 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:35.240320    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.225754287 podStartE2EDuration="2.240301384s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.547097ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:21:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:21:35.238165801 +0000 UTC m=+165.526026765" watchObservedRunningTime="2026-07-28 14:21:35.240301384 +0000 UTC m=+165.528162327"
Jul 28 14:21:36.191178 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.191148    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:21:36.191558 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.191471    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="prometheus" containerID="cri-o://d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1" gracePeriod=600
Jul 28 14:21:36.191558 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.191491    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy" containerID="cri-o://9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62" gracePeriod=600
Jul 28 14:21:36.191739 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.191522    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="thanos-sidecar" containerID="cri-o://2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1" gracePeriod=600
Jul 28 14:21:36.191796 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.191525    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12" gracePeriod=600
Jul 28 14:21:36.192195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.191551    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-web" containerID="cri-o://f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773" gracePeriod=600
Jul 28 14:21:36.192195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.191580    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="config-reloader" containerID="cri-o://14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee" gracePeriod=600
Jul 28 14:21:36.443780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.443717    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:36.561759 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561730    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwp9s\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-kube-api-access-fwp9s\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.561905 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561770    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-serving-certs-ca-bundle\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.561905 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561793    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-tls-assets\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.561905 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561815    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-db\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.561905 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561852    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-2\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.561905 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561869    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-config-out\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.561905 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561887    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-kube-rbac-proxy\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.561905 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561902    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-tls\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561926    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-trusted-ca-bundle\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561962    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-config\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.561989    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-metrics-client-certs\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-0\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562098    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562124    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-web-config\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562150    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-1\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562159    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562182    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-kubelet-serving-ca-bundle\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562237    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-grpc-tls\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562309 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562294    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-thanos-prometheus-http-client-file\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562319    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-metrics-client-ca\") pod \"be8a55bb-e9da-40d1-90a4-c3c968469c10\" (UID: \"be8a55bb-e9da-40d1-90a4-c3c968469c10\") "
Jul 28 14:21:36.562939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562597    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.562939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562314    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:36.562939 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.562611    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:36.563469 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.563441    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:21:36.563864 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.563833    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:36.564572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.564041    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:36.564572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.564245    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-config-out" (OuterVolumeSpecName: "config-out") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:21:36.564572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.564387    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:21:36.564794 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.564634    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-kube-api-access-fwp9s" (OuterVolumeSpecName: "kube-api-access-fwp9s") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "kube-api-access-fwp9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:21:36.564794 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.564638    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:36.564794 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.564639    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.565292 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.565033    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 14:21:36.565397 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.565301    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.565923 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.565894    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.566254 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.566209    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.566508 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.566486    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.566785 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.566752    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-config" (OuterVolumeSpecName: "config") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.566892 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.566870    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.566967 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.566951    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.576586 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.576566    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-web-config" (OuterVolumeSpecName: "web-config") pod "be8a55bb-e9da-40d1-90a4-c3c968469c10" (UID: "be8a55bb-e9da-40d1-90a4-c3c968469c10"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:21:36.663105 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663079    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-tls-assets\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663105 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663103    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-db\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663112    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663122    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8a55bb-e9da-40d1-90a4-c3c968469c10-config-out\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663130    2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-kube-rbac-proxy\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663139    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663147    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663157    2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663165    2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-metrics-client-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663175    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663184    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663192    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-web-config\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663201    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663209    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663238    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663246    2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-secret-grpc-tls\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663255    2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8a55bb-e9da-40d1-90a4-c3c968469c10-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663258 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663263    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8a55bb-e9da-40d1-90a4-c3c968469c10-configmap-metrics-client-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:36.663712 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:36.663271    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwp9s\" (UniqueName: \"kubernetes.io/projected/be8a55bb-e9da-40d1-90a4-c3c968469c10-kube-api-access-fwp9s\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:21:37.216782 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216753    2579 generic.go:356] "Generic (PLEG): container finished" podID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12" exitCode=0
Jul 28 14:21:37.216782 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216776    2579 generic.go:356] "Generic (PLEG): container finished" podID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62" exitCode=0
Jul 28 14:21:37.216782 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216783    2579 generic.go:356] "Generic (PLEG): container finished" podID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773" exitCode=0
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216789    2579 generic.go:356] "Generic (PLEG): container finished" podID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1" exitCode=0
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216795    2579 generic.go:356] "Generic (PLEG): container finished" podID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee" exitCode=0
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216802    2579 generic.go:356] "Generic (PLEG): container finished" podID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1" exitCode=0
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216844    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216841    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"}
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"}
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"}
Jul 28 14:21:37.217013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.216999    2579 scope.go:117] "RemoveContainer" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.217358 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.217024    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"}
Jul 28 14:21:37.217358 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.217069    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"}
Jul 28 14:21:37.217358 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.217110    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"}
Jul 28 14:21:37.217358 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.217152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8a55bb-e9da-40d1-90a4-c3c968469c10","Type":"ContainerDied","Data":"aa60008e6f702927e474c693ff210ca8a70a96cebd23a413166ed2cf2761a26e"}
Jul 28 14:21:37.224986 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.224968    2579 scope.go:117] "RemoveContainer" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.231669 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.231656    2579 scope.go:117] "RemoveContainer" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.238357 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.238343    2579 scope.go:117] "RemoveContainer" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.242697 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.242676    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:21:37.245408 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.245388    2579 scope.go:117] "RemoveContainer" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.248616 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.248597    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:21:37.252280 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.252266    2579 scope.go:117] "RemoveContainer" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.259194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.259179    2579 scope.go:117] "RemoveContainer" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.265628 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.265603    2579 scope.go:117] "RemoveContainer" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.265859 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:37.265842    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": container with ID starting with 8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12 not found: ID does not exist" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.265917 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.265866    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"} err="failed to get container status \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": rpc error: code = NotFound desc = could not find container \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": container with ID starting with 8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12 not found: ID does not exist"
Jul 28 14:21:37.265917 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.265882    2579 scope.go:117] "RemoveContainer" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.266108 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:37.266093    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": container with ID starting with 9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62 not found: ID does not exist" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.266149 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266114    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"} err="failed to get container status \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": rpc error: code = NotFound desc = could not find container \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": container with ID starting with 9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62 not found: ID does not exist"
Jul 28 14:21:37.266149 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266128    2579 scope.go:117] "RemoveContainer" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.266366 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:37.266349    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": container with ID starting with f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773 not found: ID does not exist" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.266410 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266372    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"} err="failed to get container status \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": rpc error: code = NotFound desc = could not find container \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": container with ID starting with f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773 not found: ID does not exist"
Jul 28 14:21:37.266410 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266387    2579 scope.go:117] "RemoveContainer" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.266601 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:37.266584    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": container with ID starting with 2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1 not found: ID does not exist" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.266640 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266604    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"} err="failed to get container status \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": rpc error: code = NotFound desc = could not find container \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": container with ID starting with 2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1 not found: ID does not exist"
Jul 28 14:21:37.266640 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266616    2579 scope.go:117] "RemoveContainer" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.266817 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:37.266802    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": container with ID starting with 14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee not found: ID does not exist" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.266855 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266819    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"} err="failed to get container status \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": rpc error: code = NotFound desc = could not find container \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": container with ID starting with 14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee not found: ID does not exist"
Jul 28 14:21:37.266855 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.266830    2579 scope.go:117] "RemoveContainer" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.267028 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:37.267010    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": container with ID starting with d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1 not found: ID does not exist" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.267062 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267033    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"} err="failed to get container status \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": rpc error: code = NotFound desc = could not find container \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": container with ID starting with d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1 not found: ID does not exist"
Jul 28 14:21:37.267062 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267044    2579 scope.go:117] "RemoveContainer" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.267249 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:21:37.267231    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": container with ID starting with 42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222 not found: ID does not exist" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.267287 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267252    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"} err="failed to get container status \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": rpc error: code = NotFound desc = could not find container \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": container with ID starting with 42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222 not found: ID does not exist"
Jul 28 14:21:37.267287 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267265    2579 scope.go:117] "RemoveContainer" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.267509 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267484    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"} err="failed to get container status \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": rpc error: code = NotFound desc = could not find container \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": container with ID starting with 8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12 not found: ID does not exist"
Jul 28 14:21:37.267509 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267509    2579 scope.go:117] "RemoveContainer" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.267714 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267698    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"} err="failed to get container status \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": rpc error: code = NotFound desc = could not find container \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": container with ID starting with 9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62 not found: ID does not exist"
Jul 28 14:21:37.267714 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267713    2579 scope.go:117] "RemoveContainer" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.267904 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267887    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"} err="failed to get container status \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": rpc error: code = NotFound desc = could not find container \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": container with ID starting with f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773 not found: ID does not exist"
Jul 28 14:21:37.267974 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.267907    2579 scope.go:117] "RemoveContainer" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.268144 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.268124    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"} err="failed to get container status \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": rpc error: code = NotFound desc = could not find container \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": container with ID starting with 2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1 not found: ID does not exist"
Jul 28 14:21:37.268144 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.268143    2579 scope.go:117] "RemoveContainer" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.273911 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.273881    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"} err="failed to get container status \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": rpc error: code = NotFound desc = could not find container \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": container with ID starting with 14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee not found: ID does not exist"
Jul 28 14:21:37.273998 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.273911    2579 scope.go:117] "RemoveContainer" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.274180 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274160    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"} err="failed to get container status \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": rpc error: code = NotFound desc = could not find container \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": container with ID starting with d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1 not found: ID does not exist"
Jul 28 14:21:37.274247 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274182    2579 scope.go:117] "RemoveContainer" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.274462 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274435    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"} err="failed to get container status \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": rpc error: code = NotFound desc = could not find container \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": container with ID starting with 42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222 not found: ID does not exist"
Jul 28 14:21:37.274462 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274461    2579 scope.go:117] "RemoveContainer" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.274705 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274689    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"} err="failed to get container status \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": rpc error: code = NotFound desc = could not find container \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": container with ID starting with 8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12 not found: ID does not exist"
Jul 28 14:21:37.274743 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274706    2579 scope.go:117] "RemoveContainer" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.274922 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274898    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"} err="failed to get container status \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": rpc error: code = NotFound desc = could not find container \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": container with ID starting with 9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62 not found: ID does not exist"
Jul 28 14:21:37.274922 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.274914    2579 scope.go:117] "RemoveContainer" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.275132 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.275117    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"} err="failed to get container status \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": rpc error: code = NotFound desc = could not find container \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": container with ID starting with f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773 not found: ID does not exist"
Jul 28 14:21:37.275132 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.275131    2579 scope.go:117] "RemoveContainer" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.275414 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.275369    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"} err="failed to get container status \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": rpc error: code = NotFound desc = could not find container \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": container with ID starting with 2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1 not found: ID does not exist"
Jul 28 14:21:37.275463 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.275427    2579 scope.go:117] "RemoveContainer" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.275738 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.275702    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"} err="failed to get container status \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": rpc error: code = NotFound desc = could not find container \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": container with ID starting with 14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee not found: ID does not exist"
Jul 28 14:21:37.275788 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.275745    2579 scope.go:117] "RemoveContainer" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.276029 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276014    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"} err="failed to get container status \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": rpc error: code = NotFound desc = could not find container \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": container with ID starting with d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1 not found: ID does not exist"
Jul 28 14:21:37.276029 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276029    2579 scope.go:117] "RemoveContainer" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.276268 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276250    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"} err="failed to get container status \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": rpc error: code = NotFound desc = could not find container \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": container with ID starting with 42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222 not found: ID does not exist"
Jul 28 14:21:37.276318 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276269    2579 scope.go:117] "RemoveContainer" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.276484 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276469    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"} err="failed to get container status \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": rpc error: code = NotFound desc = could not find container \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": container with ID starting with 8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12 not found: ID does not exist"
Jul 28 14:21:37.276484 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276484    2579 scope.go:117] "RemoveContainer" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.276664 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276648    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"} err="failed to get container status \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": rpc error: code = NotFound desc = could not find container \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": container with ID starting with 9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62 not found: ID does not exist"
Jul 28 14:21:37.276701 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276664    2579 scope.go:117] "RemoveContainer" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.276878 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276859    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"} err="failed to get container status \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": rpc error: code = NotFound desc = could not find container \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": container with ID starting with f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773 not found: ID does not exist"
Jul 28 14:21:37.276924 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.276880    2579 scope.go:117] "RemoveContainer" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.277067 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277048    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"} err="failed to get container status \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": rpc error: code = NotFound desc = could not find container \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": container with ID starting with 2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1 not found: ID does not exist"
Jul 28 14:21:37.277110 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277067    2579 scope.go:117] "RemoveContainer" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.277301 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277284    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"} err="failed to get container status \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": rpc error: code = NotFound desc = could not find container \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": container with ID starting with 14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee not found: ID does not exist"
Jul 28 14:21:37.277358 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277303    2579 scope.go:117] "RemoveContainer" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.277518 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277502    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"} err="failed to get container status \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": rpc error: code = NotFound desc = could not find container \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": container with ID starting with d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1 not found: ID does not exist"
Jul 28 14:21:37.277559 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277519    2579 scope.go:117] "RemoveContainer" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.277812 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277778    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"} err="failed to get container status \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": rpc error: code = NotFound desc = could not find container \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": container with ID starting with 42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222 not found: ID does not exist"
Jul 28 14:21:37.277812 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.277808    2579 scope.go:117] "RemoveContainer" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.278150 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278123    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"} err="failed to get container status \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": rpc error: code = NotFound desc = could not find container \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": container with ID starting with 8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12 not found: ID does not exist"
Jul 28 14:21:37.278150 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278150    2579 scope.go:117] "RemoveContainer" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.278430 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278407    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"} err="failed to get container status \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": rpc error: code = NotFound desc = could not find container \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": container with ID starting with 9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62 not found: ID does not exist"
Jul 28 14:21:37.278496 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278432    2579 scope.go:117] "RemoveContainer" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.278721 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278696    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"} err="failed to get container status \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": rpc error: code = NotFound desc = could not find container \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": container with ID starting with f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773 not found: ID does not exist"
Jul 28 14:21:37.278791 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278723    2579 scope.go:117] "RemoveContainer" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.278988 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278970    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"} err="failed to get container status \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": rpc error: code = NotFound desc = could not find container \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": container with ID starting with 2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1 not found: ID does not exist"
Jul 28 14:21:37.279050 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.278989    2579 scope.go:117] "RemoveContainer" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.279269 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279251    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"} err="failed to get container status \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": rpc error: code = NotFound desc = could not find container \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": container with ID starting with 14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee not found: ID does not exist"
Jul 28 14:21:37.279269 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279269    2579 scope.go:117] "RemoveContainer" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.279464 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279448    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"} err="failed to get container status \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": rpc error: code = NotFound desc = could not find container \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": container with ID starting with d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1 not found: ID does not exist"
Jul 28 14:21:37.279464 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279463    2579 scope.go:117] "RemoveContainer" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.279650 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279630    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"} err="failed to get container status \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": rpc error: code = NotFound desc = could not find container \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": container with ID starting with 42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222 not found: ID does not exist"
Jul 28 14:21:37.279713 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279651    2579 scope.go:117] "RemoveContainer" containerID="8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"
Jul 28 14:21:37.279847 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279830    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12"} err="failed to get container status \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": rpc error: code = NotFound desc = could not find container \"8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12\": container with ID starting with 8897b3b56834a9daff84c3c498f53a631e6e4ab800c3bf21eec30b09faddfb12 not found: ID does not exist"
Jul 28 14:21:37.279922 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.279847    2579 scope.go:117] "RemoveContainer" containerID="9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"
Jul 28 14:21:37.280054 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280037    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62"} err="failed to get container status \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": rpc error: code = NotFound desc = could not find container \"9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62\": container with ID starting with 9f3515aac0067112c64a2fd56ab84232293c6dcd68e964312d85a3e23cd14a62 not found: ID does not exist"
Jul 28 14:21:37.280109 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280056    2579 scope.go:117] "RemoveContainer" containerID="f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"
Jul 28 14:21:37.280185 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280168    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:21:37.280252 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280240    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773"} err="failed to get container status \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": rpc error: code = NotFound desc = could not find container \"f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773\": container with ID starting with f4cbdcb464f291426737f05346d9f73da5d42737ce90f4b5e63717e8c8493773 not found: ID does not exist"
Jul 28 14:21:37.280314 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280255    2579 scope.go:117] "RemoveContainer" containerID="2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"
Jul 28 14:21:37.280466 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280448    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1"} err="failed to get container status \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": rpc error: code = NotFound desc = could not find container \"2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1\": container with ID starting with 2d444752920e327c5dee18c9e8ba620e2a53e9dafb3989d18e6ea2aaf91f3ce1 not found: ID does not exist"
Jul 28 14:21:37.280535 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280466    2579 scope.go:117] "RemoveContainer" containerID="14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"
Jul 28 14:21:37.280626 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280611    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-web"
Jul 28 14:21:37.280687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280630    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-web"
Jul 28 14:21:37.280687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280643    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="init-config-reloader"
Jul 28 14:21:37.280687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280653    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="init-config-reloader"
Jul 28 14:21:37.280687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280661    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee"} err="failed to get container status \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": rpc error: code = NotFound desc = could not find container \"14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee\": container with ID starting with 14fc87f8fb5ac8232b39584194cec9eb26313e677cad1379b6adc43485cbe0ee not found: ID does not exist"
Jul 28 14:21:37.280687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280675    2579 scope.go:117] "RemoveContainer" containerID="d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"
Jul 28 14:21:37.280687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280678    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy"
Jul 28 14:21:37.280687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280687    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280703    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="prometheus"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280711    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="prometheus"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280720    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-thanos"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280730    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-thanos"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280740    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="thanos-sidecar"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280748    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="thanos-sidecar"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280760    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="config-reloader"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280767    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="config-reloader"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280846    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280861    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-web"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280874    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="config-reloader"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280883    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="kube-rbac-proxy-thanos"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280892    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="thanos-sidecar"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280901    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" containerName="prometheus"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280846    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1"} err="failed to get container status \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": rpc error: code = NotFound desc = could not find container \"d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1\": container with ID starting with d404f09b6d46d0806971834c71cfaeac508c52af0e19ca45fbfb8f3ef5ca27e1 not found: ID does not exist"
Jul 28 14:21:37.281013 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.280927    2579 scope.go:117] "RemoveContainer" containerID="42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"
Jul 28 14:21:37.281754 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.281094    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222"} err="failed to get container status \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": rpc error: code = NotFound desc = could not find container \"42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222\": container with ID starting with 42fa53b18709c889be886b85e83cc3f6ec40bdba2918d52a7465874d5fce2222 not found: ID does not exist"
Jul 28 14:21:37.286062 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.286046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.290463 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.290439    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 14:21:37.290559 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.290475    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 14:21:37.290559 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.290491    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 14:21:37.290671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.290491    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 14:21:37.290671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.290492    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 14:21:37.290671 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.290439    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 14:21:37.291628 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.291613    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 14:21:37.291999 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.291859    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e5p576j4h0l3o\""
Jul 28 14:21:37.291999 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.291877    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 14:21:37.291999 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.291881    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 14:21:37.291999 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.291917    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ws2b2\""
Jul 28 14:21:37.291999 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.291928    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 14:21:37.291999 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.291932    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 14:21:37.292353 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.292165    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 14:21:37.294530 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.294512    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 14:21:37.297833 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.297818    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 14:21:37.306846 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.306822    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:21:37.374824 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.374803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.374940 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.374829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.374940 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.374850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.374940 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.374895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.374956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.374982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.374998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375043 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-config\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4zs4\" (UniqueName: \"kubernetes.io/projected/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-kube-api-access-d4zs4\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375194 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375185    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375415 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375415 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375244    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375415 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375415 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375415 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.375415 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.375329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476271 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476271 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476790 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476790 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476790 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476790 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476355    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476790 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476790 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.476790 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477125 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.476877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477455 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-config\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477523 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477523 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4zs4\" (UniqueName: \"kubernetes.io/projected/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-kube-api-access-d4zs4\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.477784 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.477751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.478303 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.478280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.479174 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.479143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.479382 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.479361    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.479461 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.479358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.480075 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.479951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481353 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.480762    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481353 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.480941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481353 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481353 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481336    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481589 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-config\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481589 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481568    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481695 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481750 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481717    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481750 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.481832 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481723    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.482012 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.481994    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.482340 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.482320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.483295 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.483274    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.490865 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.490848    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4zs4\" (UniqueName: \"kubernetes.io/projected/4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f-kube-api-access-d4zs4\") pod \"prometheus-k8s-0\" (UID: \"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.595155 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.595119    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:21:37.738444 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:37.738414    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 14:21:37.740833 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:21:37.740802    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c91fa7f_4aeb_4fb3_bf06_1d3ff59b412f.slice/crio-facc84bac151bf160c93f6c650e469728acdc74056295af3523d1d01a33b4e46 WatchSource:0}: Error finding container facc84bac151bf160c93f6c650e469728acdc74056295af3523d1d01a33b4e46: Status 404 returned error can't find the container with id facc84bac151bf160c93f6c650e469728acdc74056295af3523d1d01a33b4e46
Jul 28 14:21:38.192062 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:38.192028    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be8a55bb-e9da-40d1-90a4-c3c968469c10" path="/var/lib/kubelet/pods/be8a55bb-e9da-40d1-90a4-c3c968469c10/volumes"
Jul 28 14:21:38.222424 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:38.222397    2579 generic.go:356] "Generic (PLEG): container finished" podID="4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f" containerID="2117fa11e3309a4acdea045243433e7565027f296e7b30f38a4bd19ad43a390a" exitCode=0
Jul 28 14:21:38.222543 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:38.222480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerDied","Data":"2117fa11e3309a4acdea045243433e7565027f296e7b30f38a4bd19ad43a390a"}
Jul 28 14:21:38.222599 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:38.222541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerStarted","Data":"facc84bac151bf160c93f6c650e469728acdc74056295af3523d1d01a33b4e46"}
Jul 28 14:21:39.230025 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:39.229990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerStarted","Data":"1b7afc9ef5de5f08676d97ae3bcc05fa0d8cb724d3e7d46ddb2a77cc6b867689"}
Jul 28 14:21:39.230443 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:39.230052    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerStarted","Data":"b31b226878589afb8e2aa6093d461d0df51de1a1d5160234916c2812491e5de8"}
Jul 28 14:21:39.230443 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:39.230077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerStarted","Data":"a03b08c8a0c88295f7853aae1d594dee37bd897cb870fa1c567606267798ada3"}
Jul 28 14:21:39.230443 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:39.230104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerStarted","Data":"cfbace1fd5bfe66b9158d35934a9a35dc5b988e4b4bef7314c328751339f2507"}
Jul 28 14:21:39.230443 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:39.230130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerStarted","Data":"5231ca9908a22e2d92293ca74bb3b67ed9e21a1e1f3add799b9d223f1f9c01d4"}
Jul 28 14:21:39.230443 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:39.230152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f","Type":"ContainerStarted","Data":"5d1b627d183780cd15339c70402fa144bbf707a33a46329a89fc37c189439a7f"}
Jul 28 14:21:39.268691 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:39.268641    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.249612383 podStartE2EDuration="2.268621053s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.00867ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:21:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:21:39.266198685 +0000 UTC m=+169.554059655" watchObservedRunningTime="2026-07-28 14:21:39.268621053 +0000 UTC m=+169.556481996"
Jul 28 14:21:42.596202 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:21:42.596128    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:22:37.595599 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:22:37.595563    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:22:37.610837 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:22:37.610805    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:22:38.475274 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:22:38.475228    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 14:23:50.030322 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:23:50.030292    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:23:50.034121 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:23:50.034093    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:23:50.039978 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:23:50.039961    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 14:24:02.599099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.599060    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"]
Jul 28 14:24:02.602607 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.602590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.605909 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.605881    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 14:24:02.607576 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.607558    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 14:24:02.607673 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.607647    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 14:24:02.607673 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.607654    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4xvsd\""
Jul 28 14:24:02.607799 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.607684    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 14:24:02.607799 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.607701    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 14:24:02.618355 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.618334    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"]
Jul 28 14:24:02.706426 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.706395    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-metrics-cert\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.706579 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.706451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-manager-config\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.706579 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.706495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-cert\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.706649 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.706575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnccq\" (UniqueName: \"kubernetes.io/projected/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-kube-api-access-mnccq\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.807179 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.807154    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-manager-config\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.807179 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.807183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-cert\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.807391 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.807239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnccq\" (UniqueName: \"kubernetes.io/projected/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-kube-api-access-mnccq\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.807391 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.807284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-metrics-cert\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.807928 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.807906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-manager-config\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.809561 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.809540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-cert\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.809670 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.809644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-metrics-cert\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.816833 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.816815    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnccq\" (UniqueName: \"kubernetes.io/projected/e8c2ef7f-5de6-44c4-bdb3-090998c86d72-kube-api-access-mnccq\") pod \"lws-controller-manager-5d8c94b858-xcfs8\" (UID: \"e8c2ef7f-5de6-44c4-bdb3-090998c86d72\") " pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:02.911983 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:02.911923    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:03.080288 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.080255    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"]
Jul 28 14:24:03.105577 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:24:03.105550    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c2ef7f_5de6_44c4_bdb3_090998c86d72.slice/crio-6a54979669ddff92d349376f0714648df3363517d99b3679cad15dc2bb653a38 WatchSource:0}: Error finding container 6a54979669ddff92d349376f0714648df3363517d99b3679cad15dc2bb653a38: Status 404 returned error can't find the container with id 6a54979669ddff92d349376f0714648df3363517d99b3679cad15dc2bb653a38
Jul 28 14:24:03.107542 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.107527    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:24:03.797128 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.797094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8" event={"ID":"e8c2ef7f-5de6-44c4-bdb3-090998c86d72","Type":"ContainerStarted","Data":"6a54979669ddff92d349376f0714648df3363517d99b3679cad15dc2bb653a38"}
Jul 28 14:24:03.844937 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.844906    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"]
Jul 28 14:24:03.849481 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.849467    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:03.852809 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.852779    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 14:24:03.853155 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.853129    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-htct2\""
Jul 28 14:24:03.853260 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.853167    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 14:24:03.853260 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.853223    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 14:24:03.853260 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.853210    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 14:24:03.868620 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.868602    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"]
Jul 28 14:24:03.916176 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.916134    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f59a6f01-2e62-4648-97e7-db5cd5047a2b-manifests\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:03.916335 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.916303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f59a6f01-2e62-4648-97e7-db5cd5047a2b-tmp\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:03.916390 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.916359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f59a6f01-2e62-4648-97e7-db5cd5047a2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:03.916593 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.916551    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f59a6f01-2e62-4648-97e7-db5cd5047a2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:03.916698 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:03.916678    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glncx\" (UniqueName: \"kubernetes.io/projected/f59a6f01-2e62-4648-97e7-db5cd5047a2b-kube-api-access-glncx\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.017394 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.017366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f59a6f01-2e62-4648-97e7-db5cd5047a2b-tmp\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.017394 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.017394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f59a6f01-2e62-4648-97e7-db5cd5047a2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.017603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.017422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f59a6f01-2e62-4648-97e7-db5cd5047a2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.017603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.017457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glncx\" (UniqueName: \"kubernetes.io/projected/f59a6f01-2e62-4648-97e7-db5cd5047a2b-kube-api-access-glncx\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.017603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.017485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f59a6f01-2e62-4648-97e7-db5cd5047a2b-manifests\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.017828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.017810    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f59a6f01-2e62-4648-97e7-db5cd5047a2b-manifests\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.019803 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.019776    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f59a6f01-2e62-4648-97e7-db5cd5047a2b-tmp\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.019903 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.019850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f59a6f01-2e62-4648-97e7-db5cd5047a2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.019947 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.019915    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f59a6f01-2e62-4648-97e7-db5cd5047a2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.028820 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.028796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glncx\" (UniqueName: \"kubernetes.io/projected/f59a6f01-2e62-4648-97e7-db5cd5047a2b-kube-api-access-glncx\") pod \"opendatahub-operator-controller-manager-65f7956c87-lkbmk\" (UID: \"f59a6f01-2e62-4648-97e7-db5cd5047a2b\") " pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.159124 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.159043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:04.304487 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:24:04.304455    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59a6f01_2e62_4648_97e7_db5cd5047a2b.slice/crio-3530180c7a4c76f2c08a4843bbfc0baa09598ed2f8ff88ac999b5a7dafbd0a13 WatchSource:0}: Error finding container 3530180c7a4c76f2c08a4843bbfc0baa09598ed2f8ff88ac999b5a7dafbd0a13: Status 404 returned error can't find the container with id 3530180c7a4c76f2c08a4843bbfc0baa09598ed2f8ff88ac999b5a7dafbd0a13
Jul 28 14:24:04.305844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.305819    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"]
Jul 28 14:24:04.805653 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:04.805612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk" event={"ID":"f59a6f01-2e62-4648-97e7-db5cd5047a2b","Type":"ContainerStarted","Data":"3530180c7a4c76f2c08a4843bbfc0baa09598ed2f8ff88ac999b5a7dafbd0a13"}
Jul 28 14:24:08.830512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:08.830480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk" event={"ID":"f59a6f01-2e62-4648-97e7-db5cd5047a2b","Type":"ContainerStarted","Data":"e1b3472be12f2cac0662389cdc8a602d89d76ed356bf9dc95ece4cff8c0c028c"}
Jul 28 14:24:08.832021 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:08.831999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8" event={"ID":"e8c2ef7f-5de6-44c4-bdb3-090998c86d72","Type":"ContainerStarted","Data":"5889696324fb610d0a53bf0455af563710a1ad34ccf54575bae4ebaf683eaeab"}
Jul 28 14:24:08.832151 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:08.832133    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:08.883099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:08.883048    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8" podStartSLOduration=2.049805514 podStartE2EDuration="6.883033851s" totalImagesPullingTime="4.833228337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:08.881918762 +0000 UTC m=+319.169779715" watchObservedRunningTime="2026-07-28 14:24:08.883033851 +0000 UTC m=+319.170894794"
Jul 28 14:24:09.837227 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:09.837178    2579 generic.go:356] "Generic (PLEG): container finished" podID="f59a6f01-2e62-4648-97e7-db5cd5047a2b" containerID="e1b3472be12f2cac0662389cdc8a602d89d76ed356bf9dc95ece4cff8c0c028c" exitCode=0
Jul 28 14:24:09.837703 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:09.837263    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk" event={"ID":"f59a6f01-2e62-4648-97e7-db5cd5047a2b","Type":"ContainerDied","Data":"e1b3472be12f2cac0662389cdc8a602d89d76ed356bf9dc95ece4cff8c0c028c"}
Jul 28 14:24:10.843601 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:10.843564    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk" event={"ID":"f59a6f01-2e62-4648-97e7-db5cd5047a2b","Type":"ContainerStarted","Data":"21860bfeafeeb7cca7a4f8c25e57fcff5bfa67cfd1e065bebc530e228e6c0500"}
Jul 28 14:24:10.844034 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:10.843680    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:10.868906 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:10.868862    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk" podStartSLOduration=2.644215869 podStartE2EDuration="7.868846445s" totalImagesPullingTime="4.13741945s" totalInitContainerRuntime="1.087211126s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:10.867823755 +0000 UTC m=+321.155684709" watchObservedRunningTime="2026-07-28 14:24:10.868846445 +0000 UTC m=+321.156707388"
Jul 28 14:24:19.839780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:19.839742    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5d8c94b858-xcfs8"
Jul 28 14:24:21.850107 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:21.850070    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65f7956c87-lkbmk"
Jul 28 14:24:24.305341 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.305306    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"]
Jul 28 14:24:24.351509 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.351482    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"]
Jul 28 14:24:24.351678 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.351573    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.354904 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.354880    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 14:24:24.355096 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.354959    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 14:24:24.356067 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.356048    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-k2dzl\""
Jul 28 14:24:24.356067 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.356055    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 14:24:24.356282 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.356059    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 14:24:24.499901 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.499873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-tmp\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.500042 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.499910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbtqk\" (UniqueName: \"kubernetes.io/projected/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-kube-api-access-tbtqk\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.500042 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.499954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-tls-certs\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.601154 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.601079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-tmp\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.601154 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.601121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbtqk\" (UniqueName: \"kubernetes.io/projected/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-kube-api-access-tbtqk\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.601401 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.601180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-tls-certs\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.603541 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.603512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-tls-certs\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.603673 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.603591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-tmp\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.615305 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.615279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbtqk\" (UniqueName: \"kubernetes.io/projected/e2d81f2b-8cd7-466d-9b4e-ec7894c2c659-kube-api-access-tbtqk\") pod \"kube-auth-proxy-65c4fff746-25fcb\" (UID: \"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659\") " pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.660718 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.660693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"
Jul 28 14:24:24.836149 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.836113    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c4fff746-25fcb"]
Jul 28 14:24:24.837122 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:24:24.837092    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d81f2b_8cd7_466d_9b4e_ec7894c2c659.slice/crio-0411e759ab9df02c46541c86166585b60565eb1e5feac275bcbb6c734fb86c37 WatchSource:0}: Error finding container 0411e759ab9df02c46541c86166585b60565eb1e5feac275bcbb6c734fb86c37: Status 404 returned error can't find the container with id 0411e759ab9df02c46541c86166585b60565eb1e5feac275bcbb6c734fb86c37
Jul 28 14:24:24.905883 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:24.905808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb" event={"ID":"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659","Type":"ContainerStarted","Data":"0411e759ab9df02c46541c86166585b60565eb1e5feac275bcbb6c734fb86c37"}
Jul 28 14:24:27.920083 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:27.919994    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb" event={"ID":"e2d81f2b-8cd7-466d-9b4e-ec7894c2c659","Type":"ContainerStarted","Data":"da2c3bd517def04a5aede0944246e5ac284ac8e7b36ba25c5d729a5c79955e6f"}
Jul 28 14:24:27.938482 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:24:27.938429    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65c4fff746-25fcb" podStartSLOduration=1.21474534 podStartE2EDuration="3.938415797s" totalImagesPullingTime="2.723670457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:27.937092426 +0000 UTC m=+338.224953404" watchObservedRunningTime="2026-07-28 14:24:27.938415797 +0000 UTC m=+338.226276740"
Jul 28 14:26:06.628859 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.628823    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"]
Jul 28 14:26:06.632388 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.632371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:06.635610 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.635585    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 14:26:06.635730 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.635643    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 14:26:06.635730 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.635667    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dnzzf\""
Jul 28 14:26:06.658535 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.658511    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"]
Jul 28 14:26:06.777333 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.777298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfac4552-67bb-4cd6-b6b2-a10899d80b71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:06.777499 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.777352    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdtld\" (UniqueName: \"kubernetes.io/projected/dfac4552-67bb-4cd6-b6b2-a10899d80b71-kube-api-access-rdtld\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:06.878469 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.878434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfac4552-67bb-4cd6-b6b2-a10899d80b71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:06.878636 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.878491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdtld\" (UniqueName: \"kubernetes.io/projected/dfac4552-67bb-4cd6-b6b2-a10899d80b71-kube-api-access-rdtld\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:06.878851 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.878831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfac4552-67bb-4cd6-b6b2-a10899d80b71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:06.889759 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.889712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdtld\" (UniqueName: \"kubernetes.io/projected/dfac4552-67bb-4cd6-b6b2-a10899d80b71-kube-api-access-rdtld\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:06.944305 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:06.944273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:07.109726 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:07.109701    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"]
Jul 28 14:26:07.112236 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:26:07.112189    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfac4552_67bb_4cd6_b6b2_a10899d80b71.slice/crio-ab174b6d5fcc435ffb35941f1de6ec3ef2906c81f42de9b79e74ccf5b23ea23d WatchSource:0}: Error finding container ab174b6d5fcc435ffb35941f1de6ec3ef2906c81f42de9b79e74ccf5b23ea23d: Status 404 returned error can't find the container with id ab174b6d5fcc435ffb35941f1de6ec3ef2906c81f42de9b79e74ccf5b23ea23d
Jul 28 14:26:07.344254 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:07.344200    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq" event={"ID":"dfac4552-67bb-4cd6-b6b2-a10899d80b71","Type":"ContainerStarted","Data":"ab174b6d5fcc435ffb35941f1de6ec3ef2906c81f42de9b79e74ccf5b23ea23d"}
Jul 28 14:26:14.377156 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:14.377119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq" event={"ID":"dfac4552-67bb-4cd6-b6b2-a10899d80b71","Type":"ContainerStarted","Data":"65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1"}
Jul 28 14:26:14.377664 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:14.377198    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:14.412899 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:14.412854    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq" podStartSLOduration=1.720485632 podStartE2EDuration="8.41284075s" totalImagesPullingTime="6.692355118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:26:14.41128377 +0000 UTC m=+444.699144731" watchObservedRunningTime="2026-07-28 14:26:14.41284075 +0000 UTC m=+444.700701693"
Jul 28 14:26:25.384126 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:25.384093    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:28.001177 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.001146    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"]
Jul 28 14:26:28.001625 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.001333    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq" podUID="dfac4552-67bb-4cd6-b6b2-a10899d80b71" containerName="manager" containerID="cri-o://65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1" gracePeriod=2
Jul 28 14:26:28.030653 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.030623    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"]
Jul 28 14:26:28.049055 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:26:28.049020    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"dfac4552-67bb-4cd6-b6b2-a10899d80b71\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\": pods \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:28.158157 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.158116    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"]
Jul 28 14:26:28.158490 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.158473    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfac4552-67bb-4cd6-b6b2-a10899d80b71" containerName="manager"
Jul 28 14:26:28.158490 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.158489    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfac4552-67bb-4cd6-b6b2-a10899d80b71" containerName="manager"
Jul 28 14:26:28.158626 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.158571    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfac4552-67bb-4cd6-b6b2-a10899d80b71" containerName="manager"
Jul 28 14:26:28.162279 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.162260    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"
Jul 28 14:26:28.166453 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.166434    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-p57sk\""
Jul 28 14:26:28.198347 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.198319    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"]
Jul 28 14:26:28.224379 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:26:28.224349    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wn8wq\" is forbidden: User \"system:node:ip-10-0-141-50.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-50.ec2.internal' and this object" podUID="dfac4552-67bb-4cd6-b6b2-a10899d80b71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:28.263482 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.263457    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpszf\" (UniqueName: \"kubernetes.io/projected/a91dbff4-a01e-4086-acb6-d95e1180837c-kube-api-access-hpszf\") pod \"limitador-operator-controller-manager-859d7b78bd-5j4fh\" (UID: \"a91dbff4-a01e-4086-acb6-d95e1180837c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"
Jul 28 14:26:28.276271 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.276252    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:28.364467 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.364446    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdtld\" (UniqueName: \"kubernetes.io/projected/dfac4552-67bb-4cd6-b6b2-a10899d80b71-kube-api-access-rdtld\") pod \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") "
Jul 28 14:26:28.364598 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.364512    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfac4552-67bb-4cd6-b6b2-a10899d80b71-extensions-socket-volume\") pod \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\" (UID: \"dfac4552-67bb-4cd6-b6b2-a10899d80b71\") "
Jul 28 14:26:28.364598 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.364587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpszf\" (UniqueName: \"kubernetes.io/projected/a91dbff4-a01e-4086-acb6-d95e1180837c-kube-api-access-hpszf\") pod \"limitador-operator-controller-manager-859d7b78bd-5j4fh\" (UID: \"a91dbff4-a01e-4086-acb6-d95e1180837c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"
Jul 28 14:26:28.364987 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.364965    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfac4552-67bb-4cd6-b6b2-a10899d80b71-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dfac4552-67bb-4cd6-b6b2-a10899d80b71" (UID: "dfac4552-67bb-4cd6-b6b2-a10899d80b71"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:26:28.366657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.366632    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfac4552-67bb-4cd6-b6b2-a10899d80b71-kube-api-access-rdtld" (OuterVolumeSpecName: "kube-api-access-rdtld") pod "dfac4552-67bb-4cd6-b6b2-a10899d80b71" (UID: "dfac4552-67bb-4cd6-b6b2-a10899d80b71"). InnerVolumeSpecName "kube-api-access-rdtld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:26:28.393087 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.393057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpszf\" (UniqueName: \"kubernetes.io/projected/a91dbff4-a01e-4086-acb6-d95e1180837c-kube-api-access-hpszf\") pod \"limitador-operator-controller-manager-859d7b78bd-5j4fh\" (UID: \"a91dbff4-a01e-4086-acb6-d95e1180837c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"
Jul 28 14:26:28.441033 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.441006    2579 generic.go:356] "Generic (PLEG): container finished" podID="dfac4552-67bb-4cd6-b6b2-a10899d80b71" containerID="65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1" exitCode=0
Jul 28 14:26:28.441137 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.441048    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wn8wq"
Jul 28 14:26:28.441137 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.441059    2579 scope.go:117] "RemoveContainer" containerID="65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1"
Jul 28 14:26:28.449993 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.449970    2579 scope.go:117] "RemoveContainer" containerID="65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1"
Jul 28 14:26:28.450255 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:26:28.450237    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1\": container with ID starting with 65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1 not found: ID does not exist" containerID="65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1"
Jul 28 14:26:28.450322 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.450263    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1"} err="failed to get container status \"65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1\": rpc error: code = NotFound desc = could not find container \"65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1\": container with ID starting with 65a6d43fdc729ab0aa534fa1ea496c77ca14baf135d159152564101d42217ff1 not found: ID does not exist"
Jul 28 14:26:28.466095 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.466078    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfac4552-67bb-4cd6-b6b2-a10899d80b71-extensions-socket-volume\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:26:28.466167 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.466096    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdtld\" (UniqueName: \"kubernetes.io/projected/dfac4552-67bb-4cd6-b6b2-a10899d80b71-kube-api-access-rdtld\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:26:28.471984 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.471964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"
Jul 28 14:26:28.643242 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:28.643206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"]
Jul 28 14:26:28.645923 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:26:28.645895    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91dbff4_a01e_4086_acb6_d95e1180837c.slice/crio-a60359cae0917675784ad9e63028f171283532dbca7de1d46ee156f5ef567a1e WatchSource:0}: Error finding container a60359cae0917675784ad9e63028f171283532dbca7de1d46ee156f5ef567a1e: Status 404 returned error can't find the container with id a60359cae0917675784ad9e63028f171283532dbca7de1d46ee156f5ef567a1e
Jul 28 14:26:29.449575 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:29.449529    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh" event={"ID":"a91dbff4-a01e-4086-acb6-d95e1180837c","Type":"ContainerStarted","Data":"a60359cae0917675784ad9e63028f171283532dbca7de1d46ee156f5ef567a1e"}
Jul 28 14:26:30.191894 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.191863    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfac4552-67bb-4cd6-b6b2-a10899d80b71" path="/var/lib/kubelet/pods/dfac4552-67bb-4cd6-b6b2-a10899d80b71/volumes"
Jul 28 14:26:30.456047 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.456011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh" event={"ID":"a91dbff4-a01e-4086-acb6-d95e1180837c","Type":"ContainerStarted","Data":"72225ba9577a11538c66dcda3eb1d482fa480b7d1562d2a6dcf2b1c4b12d2707"}
Jul 28 14:26:30.456522 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.456244    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"
Jul 28 14:26:30.481458 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.481417    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh" podStartSLOduration=1.114279563 podStartE2EDuration="2.481402273s" totalImagesPullingTime="1.36712271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:26:30.479169941 +0000 UTC m=+460.767030885" watchObservedRunningTime="2026-07-28 14:26:30.481402273 +0000 UTC m=+460.769263215"
Jul 28 14:26:30.658304 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.658277    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"]
Jul 28 14:26:30.661814 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.661797    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:30.671562 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.671543    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dnzzf\""
Jul 28 14:26:30.684695 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.684676    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"]
Jul 28 14:26:30.787388 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.787298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk4qj\" (UniqueName: \"kubernetes.io/projected/74948eba-4164-43eb-833f-61dab994c033-kube-api-access-kk4qj\") pod \"kuadrant-operator-controller-manager-86878c769f-jth66\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:30.787388 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.787369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74948eba-4164-43eb-833f-61dab994c033-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jth66\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:30.888087 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.888056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk4qj\" (UniqueName: \"kubernetes.io/projected/74948eba-4164-43eb-833f-61dab994c033-kube-api-access-kk4qj\") pod \"kuadrant-operator-controller-manager-86878c769f-jth66\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:30.888242 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.888108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74948eba-4164-43eb-833f-61dab994c033-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jth66\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:30.888464 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.888443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74948eba-4164-43eb-833f-61dab994c033-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jth66\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:30.920783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.920757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk4qj\" (UniqueName: \"kubernetes.io/projected/74948eba-4164-43eb-833f-61dab994c033-kube-api-access-kk4qj\") pod \"kuadrant-operator-controller-manager-86878c769f-jth66\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:30.971602 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:30.971579    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:31.143783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:31.143753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"]
Jul 28 14:26:31.154368 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:26:31.154345    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74948eba_4164_43eb_833f_61dab994c033.slice/crio-ac45f882363a179eb8e7a510ca962fa88313f246f47e86ea4b9eb57718936497 WatchSource:0}: Error finding container ac45f882363a179eb8e7a510ca962fa88313f246f47e86ea4b9eb57718936497: Status 404 returned error can't find the container with id ac45f882363a179eb8e7a510ca962fa88313f246f47e86ea4b9eb57718936497
Jul 28 14:26:31.462854 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:31.462822    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66" event={"ID":"74948eba-4164-43eb-833f-61dab994c033","Type":"ContainerStarted","Data":"0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9"}
Jul 28 14:26:31.463205 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:31.462870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66" event={"ID":"74948eba-4164-43eb-833f-61dab994c033","Type":"ContainerStarted","Data":"ac45f882363a179eb8e7a510ca962fa88313f246f47e86ea4b9eb57718936497"}
Jul 28 14:26:31.525948 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:31.525903    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66" podStartSLOduration=1.525888535 podStartE2EDuration="1.525888535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:26:31.525363403 +0000 UTC m=+461.813224346" watchObservedRunningTime="2026-07-28 14:26:31.525888535 +0000 UTC m=+461.813749478"
Jul 28 14:26:32.467559 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:32.467527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:26:41.464733 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:41.464705    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5j4fh"
Jul 28 14:26:43.474550 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:26:43.474511    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:27:02.978990 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:02.978953    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-n2jnq"]
Jul 28 14:27:02.985582 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:02.985565    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n2jnq"
Jul 28 14:27:02.990071 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:02.990047    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-49ffk\""
Jul 28 14:27:02.990912 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:02.990888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n2jnq"]
Jul 28 14:27:03.162245 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:03.162206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvfn2\" (UniqueName: \"kubernetes.io/projected/eca13b3c-2f00-4a44-8dc5-26f905a6848a-kube-api-access-vvfn2\") pod \"authorino-75d556bd77-n2jnq\" (UID: \"eca13b3c-2f00-4a44-8dc5-26f905a6848a\") " pod="kuadrant-system/authorino-75d556bd77-n2jnq"
Jul 28 14:27:03.263701 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:03.263617    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvfn2\" (UniqueName: \"kubernetes.io/projected/eca13b3c-2f00-4a44-8dc5-26f905a6848a-kube-api-access-vvfn2\") pod \"authorino-75d556bd77-n2jnq\" (UID: \"eca13b3c-2f00-4a44-8dc5-26f905a6848a\") " pod="kuadrant-system/authorino-75d556bd77-n2jnq"
Jul 28 14:27:03.283487 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:03.283458    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvfn2\" (UniqueName: \"kubernetes.io/projected/eca13b3c-2f00-4a44-8dc5-26f905a6848a-kube-api-access-vvfn2\") pod \"authorino-75d556bd77-n2jnq\" (UID: \"eca13b3c-2f00-4a44-8dc5-26f905a6848a\") " pod="kuadrant-system/authorino-75d556bd77-n2jnq"
Jul 28 14:27:03.295227 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:03.295199    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n2jnq"
Jul 28 14:27:03.429997 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:03.429973    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n2jnq"]
Jul 28 14:27:03.432169 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:27:03.432144    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeca13b3c_2f00_4a44_8dc5_26f905a6848a.slice/crio-3fea044bf7ea9fd73bdae55f0a2fd351e2c0d79ffc22c1495217fb3e3a46b31e WatchSource:0}: Error finding container 3fea044bf7ea9fd73bdae55f0a2fd351e2c0d79ffc22c1495217fb3e3a46b31e: Status 404 returned error can't find the container with id 3fea044bf7ea9fd73bdae55f0a2fd351e2c0d79ffc22c1495217fb3e3a46b31e
Jul 28 14:27:03.611071 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:03.610994    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n2jnq" event={"ID":"eca13b3c-2f00-4a44-8dc5-26f905a6848a","Type":"ContainerStarted","Data":"3fea044bf7ea9fd73bdae55f0a2fd351e2c0d79ffc22c1495217fb3e3a46b31e"}
Jul 28 14:27:07.633367 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:07.633326    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n2jnq" event={"ID":"eca13b3c-2f00-4a44-8dc5-26f905a6848a","Type":"ContainerStarted","Data":"51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d"}
Jul 28 14:27:07.652919 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:07.652876    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-n2jnq" podStartSLOduration=2.2701433890000002 podStartE2EDuration="5.652861487s" totalImagesPullingTime="3.382718098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:27:07.651062544 +0000 UTC m=+497.938923487" watchObservedRunningTime="2026-07-28 14:27:07.652861487 +0000 UTC m=+497.940722470"
Jul 28 14:27:44.080969 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.080893    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n2jnq"]
Jul 28 14:27:44.081395 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.081082    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-n2jnq" podUID="eca13b3c-2f00-4a44-8dc5-26f905a6848a" containerName="authorino" containerID="cri-o://51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d" gracePeriod=30
Jul 28 14:27:44.327687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.327649    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n2jnq"
Jul 28 14:27:44.396236 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.396142    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvfn2\" (UniqueName: \"kubernetes.io/projected/eca13b3c-2f00-4a44-8dc5-26f905a6848a-kube-api-access-vvfn2\") pod \"eca13b3c-2f00-4a44-8dc5-26f905a6848a\" (UID: \"eca13b3c-2f00-4a44-8dc5-26f905a6848a\") "
Jul 28 14:27:44.398093 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.398062    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca13b3c-2f00-4a44-8dc5-26f905a6848a-kube-api-access-vvfn2" (OuterVolumeSpecName: "kube-api-access-vvfn2") pod "eca13b3c-2f00-4a44-8dc5-26f905a6848a" (UID: "eca13b3c-2f00-4a44-8dc5-26f905a6848a"). InnerVolumeSpecName "kube-api-access-vvfn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:27:44.497004 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.496970    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvfn2\" (UniqueName: \"kubernetes.io/projected/eca13b3c-2f00-4a44-8dc5-26f905a6848a-kube-api-access-vvfn2\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:27:44.801233 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.801175    2579 generic.go:356] "Generic (PLEG): container finished" podID="eca13b3c-2f00-4a44-8dc5-26f905a6848a" containerID="51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d" exitCode=0
Jul 28 14:27:44.801233 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.801238    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n2jnq"
Jul 28 14:27:44.801432 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.801261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n2jnq" event={"ID":"eca13b3c-2f00-4a44-8dc5-26f905a6848a","Type":"ContainerDied","Data":"51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d"}
Jul 28 14:27:44.801432 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.801318    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n2jnq" event={"ID":"eca13b3c-2f00-4a44-8dc5-26f905a6848a","Type":"ContainerDied","Data":"3fea044bf7ea9fd73bdae55f0a2fd351e2c0d79ffc22c1495217fb3e3a46b31e"}
Jul 28 14:27:44.801432 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.801335    2579 scope.go:117] "RemoveContainer" containerID="51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d"
Jul 28 14:27:44.809956 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.809937    2579 scope.go:117] "RemoveContainer" containerID="51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d"
Jul 28 14:27:44.810231 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:27:44.810196    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d\": container with ID starting with 51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d not found: ID does not exist" containerID="51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d"
Jul 28 14:27:44.810281 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.810247    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d"} err="failed to get container status \"51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d\": rpc error: code = NotFound desc = could not find container \"51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d\": container with ID starting with 51e667dac3cfa28d7e5f07cf5f6bfdf53a49ba77a012859b6f61306ca49d625d not found: ID does not exist"
Jul 28 14:27:44.828546 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.828525    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n2jnq"]
Jul 28 14:27:44.836503 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:44.836483    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n2jnq"]
Jul 28 14:27:46.192535 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:27:46.192504    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca13b3c-2f00-4a44-8dc5-26f905a6848a" path="/var/lib/kubelet/pods/eca13b3c-2f00-4a44-8dc5-26f905a6848a/volumes"
Jul 28 14:28:50.062232 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:28:50.062192    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:28:50.062662 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:28:50.062304    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:29:12.768193 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.768112    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"]
Jul 28 14:29:12.770388 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.768687    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca13b3c-2f00-4a44-8dc5-26f905a6848a" containerName="authorino"
Jul 28 14:29:12.770388 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.768701    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca13b3c-2f00-4a44-8dc5-26f905a6848a" containerName="authorino"
Jul 28 14:29:12.770388 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.768758    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca13b3c-2f00-4a44-8dc5-26f905a6848a" containerName="authorino"
Jul 28 14:29:12.771744 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.771726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.774877 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.774858    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 14:29:12.777108 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.777089    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2p4xs\""
Jul 28 14:29:12.777436 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.777422    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 14:29:12.778596 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.778577    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 14:29:12.783613 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.783591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"]
Jul 28 14:29:12.850785 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.850746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d46779a-f1cd-44fd-9b07-129276142eae-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.850785 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.850783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75p6\" (UniqueName: \"kubernetes.io/projected/8d46779a-f1cd-44fd-9b07-129276142eae-kube-api-access-j75p6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.851002 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.850848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.851002 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.850875    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.851002 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.850899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.851002 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.850937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.951657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.951622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.951657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.951662    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.951844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.951679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.951844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.951721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.951844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.951804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d46779a-f1cd-44fd-9b07-129276142eae-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.951844 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.951826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75p6\" (UniqueName: \"kubernetes.io/projected/8d46779a-f1cd-44fd-9b07-129276142eae-kube-api-access-j75p6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.952071 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.952052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.952138 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.952106    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.952286 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.952266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.953860 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.953838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d46779a-f1cd-44fd-9b07-129276142eae-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.954141 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.954122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d46779a-f1cd-44fd-9b07-129276142eae-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:12.961928 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:12.961908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75p6\" (UniqueName: \"kubernetes.io/projected/8d46779a-f1cd-44fd-9b07-129276142eae-kube-api-access-j75p6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-swbxm\" (UID: \"8d46779a-f1cd-44fd-9b07-129276142eae\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:13.083534 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:13.083457    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:13.217165 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:13.217143    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"]
Jul 28 14:29:13.219609 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:29:13.219582    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d46779a_f1cd_44fd_9b07_129276142eae.slice/crio-62a2000ed3e1e85e46d49fba2dbc4c256d78bd6138427a18a0d30118a5f223b9 WatchSource:0}: Error finding container 62a2000ed3e1e85e46d49fba2dbc4c256d78bd6138427a18a0d30118a5f223b9: Status 404 returned error can't find the container with id 62a2000ed3e1e85e46d49fba2dbc4c256d78bd6138427a18a0d30118a5f223b9
Jul 28 14:29:13.220873 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:13.220848    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 14:29:14.237603 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:14.237528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm" event={"ID":"8d46779a-f1cd-44fd-9b07-129276142eae","Type":"ContainerStarted","Data":"62a2000ed3e1e85e46d49fba2dbc4c256d78bd6138427a18a0d30118a5f223b9"}
Jul 28 14:29:15.244174 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:15.244136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm" event={"ID":"8d46779a-f1cd-44fd-9b07-129276142eae","Type":"ContainerStarted","Data":"3e91369921b13b49f7b30cd740f9425721c5001b7b61a346f488f1544f578c48"}
Jul 28 14:29:15.268047 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:15.268001    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm" podStartSLOduration=2.187345947 podStartE2EDuration="3.267984557s" totalImagesPullingTime="1.08063861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:29:15.265348216 +0000 UTC m=+625.553209209" watchObservedRunningTime="2026-07-28 14:29:15.267984557 +0000 UTC m=+625.555845501"
Jul 28 14:29:23.084050 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:23.084009    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:23.084523 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:23.084078    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:23.096295 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:23.096274    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:23.290621 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:23.290596    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-swbxm"
Jul 28 14:29:58.883933 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.883900    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"]
Jul 28 14:29:58.887574 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.887559    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:58.891375 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.891346    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 14:29:58.908556 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.908531    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"]
Jul 28 14:29:58.961886 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.961854    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42ae94c2-5ee5-4967-acb9-ac73aca7402d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:58.962051 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.961895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:58.962051 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.961913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vprx9\" (UniqueName: \"kubernetes.io/projected/42ae94c2-5ee5-4967-acb9-ac73aca7402d-kube-api-access-vprx9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:58.962051 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.962021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:58.962195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.962057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:58.962195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:58.962173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063235 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42ae94c2-5ee5-4967-acb9-ac73aca7402d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063410 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063410 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprx9\" (UniqueName: \"kubernetes.io/projected/42ae94c2-5ee5-4967-acb9-ac73aca7402d-kube-api-access-vprx9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063410 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063410 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063970 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063970 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063970 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063779    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.063970 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.063834    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.065428 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.065406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42ae94c2-5ee5-4967-acb9-ac73aca7402d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.065780 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.065763    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42ae94c2-5ee5-4967-acb9-ac73aca7402d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.073389 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.073360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vprx9\" (UniqueName: \"kubernetes.io/projected/42ae94c2-5ee5-4967-acb9-ac73aca7402d-kube-api-access-vprx9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-29k2r\" (UID: \"42ae94c2-5ee5-4967-acb9-ac73aca7402d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.199668 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.199635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:29:59.338714 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.338677    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"]
Jul 28 14:29:59.340592 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:29:59.340570    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42ae94c2_5ee5_4967_acb9_ac73aca7402d.slice/crio-8b01dfe2aa1ae375bcbfff4737d1a68b6f3d0fa53751589b5ddf9b0a04d10c24 WatchSource:0}: Error finding container 8b01dfe2aa1ae375bcbfff4737d1a68b6f3d0fa53751589b5ddf9b0a04d10c24: Status 404 returned error can't find the container with id 8b01dfe2aa1ae375bcbfff4737d1a68b6f3d0fa53751589b5ddf9b0a04d10c24
Jul 28 14:29:59.452118 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:29:59.452038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r" event={"ID":"42ae94c2-5ee5-4967-acb9-ac73aca7402d","Type":"ContainerStarted","Data":"8b01dfe2aa1ae375bcbfff4737d1a68b6f3d0fa53751589b5ddf9b0a04d10c24"}
Jul 28 14:30:00.458308 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:00.458270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r" event={"ID":"42ae94c2-5ee5-4967-acb9-ac73aca7402d","Type":"ContainerStarted","Data":"30763e421d97bf1035685def0475d5d1444c03c4586adc238a4f42496d4215d4"}
Jul 28 14:30:00.482556 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:00.482511    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r" podStartSLOduration=2.193163336 podStartE2EDuration="2.482495427s" totalImagesPullingTime="289.332091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:30:00.482348548 +0000 UTC m=+670.770209491" watchObservedRunningTime="2026-07-28 14:30:00.482495427 +0000 UTC m=+670.770356369"
Jul 28 14:30:09.200039 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:09.200004    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:30:09.200452 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:09.200055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:30:09.212184 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:09.212158    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:30:09.510023 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:09.509950    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-29k2r"
Jul 28 14:30:58.658572 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.658494    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6988c6fd4-5qhnh"]
Jul 28 14:30:58.661891 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.661874    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:58.668316 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.668296    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 14:30:58.668444 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.668327    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-49ffk\""
Jul 28 14:30:58.671677 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.671656    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6988c6fd4-5qhnh"]
Jul 28 14:30:58.802811 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.802774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnjtw\" (UniqueName: \"kubernetes.io/projected/8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3-kube-api-access-pnjtw\") pod \"authorino-6988c6fd4-5qhnh\" (UID: \"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3\") " pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:58.802978 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.802829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3-tls-cert\") pod \"authorino-6988c6fd4-5qhnh\" (UID: \"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3\") " pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:58.904306 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.904274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnjtw\" (UniqueName: \"kubernetes.io/projected/8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3-kube-api-access-pnjtw\") pod \"authorino-6988c6fd4-5qhnh\" (UID: \"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3\") " pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:58.904474 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.904333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3-tls-cert\") pod \"authorino-6988c6fd4-5qhnh\" (UID: \"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3\") " pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:58.906662 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.906635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3-tls-cert\") pod \"authorino-6988c6fd4-5qhnh\" (UID: \"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3\") " pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:58.913381 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.913309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnjtw\" (UniqueName: \"kubernetes.io/projected/8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3-kube-api-access-pnjtw\") pod \"authorino-6988c6fd4-5qhnh\" (UID: \"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3\") " pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:58.970815 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:58.970776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6988c6fd4-5qhnh"
Jul 28 14:30:59.116630 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:59.116605    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6988c6fd4-5qhnh"]
Jul 28 14:30:59.119184 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:30:59.119159    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a9f8324_1c9e_43a5_ab2d_ec2b3cf3d6e3.slice/crio-88beb115b069edf1aab79cad08d40526cf17ce975566158789d32c9f6885f23b WatchSource:0}: Error finding container 88beb115b069edf1aab79cad08d40526cf17ce975566158789d32c9f6885f23b: Status 404 returned error can't find the container with id 88beb115b069edf1aab79cad08d40526cf17ce975566158789d32c9f6885f23b
Jul 28 14:30:59.733154 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:30:59.733127    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6988c6fd4-5qhnh" event={"ID":"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3","Type":"ContainerStarted","Data":"88beb115b069edf1aab79cad08d40526cf17ce975566158789d32c9f6885f23b"}
Jul 28 14:31:00.739870 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:31:00.739834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6988c6fd4-5qhnh" event={"ID":"8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3","Type":"ContainerStarted","Data":"cc928f8f23f13353cf82c78890847d7b2d7db158b55fb73a547240a0d8b13fb1"}
Jul 28 14:33:50.092830 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:33:50.092762    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:33:50.094166 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:33:50.094141    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:38:21.121007 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:21.120909    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6988c6fd4-5qhnh" podStartSLOduration=442.517602863 podStartE2EDuration="7m23.120891991s" totalImagesPullingTime="603.289128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:31:00.773257614 +0000 UTC m=+731.061118555" watchObservedRunningTime="2026-07-28 14:38:21.120891991 +0000 UTC m=+1171.408752933"
Jul 28 14:38:21.121534 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:21.121033    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"]
Jul 28 14:38:21.121534 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:21.121258    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66" podUID="74948eba-4164-43eb-833f-61dab994c033" containerName="manager" containerID="cri-o://0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9" gracePeriod=10
Jul 28 14:38:24.369254 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.369206    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:38:24.422412 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.422340    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk4qj\" (UniqueName: \"kubernetes.io/projected/74948eba-4164-43eb-833f-61dab994c033-kube-api-access-kk4qj\") pod \"74948eba-4164-43eb-833f-61dab994c033\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") "
Jul 28 14:38:24.422557 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.422500    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74948eba-4164-43eb-833f-61dab994c033-extensions-socket-volume\") pod \"74948eba-4164-43eb-833f-61dab994c033\" (UID: \"74948eba-4164-43eb-833f-61dab994c033\") "
Jul 28 14:38:24.422961 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.422937    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74948eba-4164-43eb-833f-61dab994c033-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "74948eba-4164-43eb-833f-61dab994c033" (UID: "74948eba-4164-43eb-833f-61dab994c033"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:38:24.425106 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.425076    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74948eba-4164-43eb-833f-61dab994c033-kube-api-access-kk4qj" (OuterVolumeSpecName: "kube-api-access-kk4qj") pod "74948eba-4164-43eb-833f-61dab994c033" (UID: "74948eba-4164-43eb-833f-61dab994c033"). InnerVolumeSpecName "kube-api-access-kk4qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:38:24.524074 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.524041    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74948eba-4164-43eb-833f-61dab994c033-extensions-socket-volume\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:38:24.524074 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.524070    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk4qj\" (UniqueName: \"kubernetes.io/projected/74948eba-4164-43eb-833f-61dab994c033-kube-api-access-kk4qj\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:38:24.888957 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.888920    2579 generic.go:356] "Generic (PLEG): container finished" podID="74948eba-4164-43eb-833f-61dab994c033" containerID="0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9" exitCode=0
Jul 28 14:38:24.889158 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.888962    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"
Jul 28 14:38:24.889158 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.888976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66" event={"ID":"74948eba-4164-43eb-833f-61dab994c033","Type":"ContainerDied","Data":"0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9"}
Jul 28 14:38:24.889158 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.889066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66" event={"ID":"74948eba-4164-43eb-833f-61dab994c033","Type":"ContainerDied","Data":"ac45f882363a179eb8e7a510ca962fa88313f246f47e86ea4b9eb57718936497"}
Jul 28 14:38:24.889158 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.889099    2579 scope.go:117] "RemoveContainer" containerID="0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9"
Jul 28 14:38:24.899825 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.899808    2579 scope.go:117] "RemoveContainer" containerID="0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9"
Jul 28 14:38:24.900107 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:38:24.900083    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9\": container with ID starting with 0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9 not found: ID does not exist" containerID="0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9"
Jul 28 14:38:24.900200 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.900113    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9"} err="failed to get container status \"0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9\": rpc error: code = NotFound desc = could not find container \"0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9\": container with ID starting with 0e2d3f08b582922ea9b5581e200ac2b59f636876bf95a40e95ce13f673cff8d9 not found: ID does not exist"
Jul 28 14:38:24.910692 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.910667    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"]
Jul 28 14:38:24.914489 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:24.914471    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jth66"]
Jul 28 14:38:26.192729 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:26.192695    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74948eba-4164-43eb-833f-61dab994c033" path="/var/lib/kubelet/pods/74948eba-4164-43eb-833f-61dab994c033/volumes"
Jul 28 14:38:50.121816 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:50.121785    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:38:50.124492 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:38:50.124470    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:39:22.345375 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.345342    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"]
Jul 28 14:39:22.345830 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.345724    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74948eba-4164-43eb-833f-61dab994c033" containerName="manager"
Jul 28 14:39:22.345830 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.345740    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74948eba-4164-43eb-833f-61dab994c033" containerName="manager"
Jul 28 14:39:22.345830 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.345811    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74948eba-4164-43eb-833f-61dab994c033" containerName="manager"
Jul 28 14:39:22.348744 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.348729    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.351657 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.351639    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dnzzf\""
Jul 28 14:39:22.359769 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.359750    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"]
Jul 28 14:39:22.431772 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.431747    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff9a922-cd91-4600-988c-276b1a848b0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wkpq4\" (UID: \"2ff9a922-cd91-4600-988c-276b1a848b0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.431887 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.431804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ls4l\" (UniqueName: \"kubernetes.io/projected/2ff9a922-cd91-4600-988c-276b1a848b0e-kube-api-access-8ls4l\") pod \"kuadrant-operator-controller-manager-86878c769f-wkpq4\" (UID: \"2ff9a922-cd91-4600-988c-276b1a848b0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.532356 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.532314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff9a922-cd91-4600-988c-276b1a848b0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wkpq4\" (UID: \"2ff9a922-cd91-4600-988c-276b1a848b0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.532473 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.532394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ls4l\" (UniqueName: \"kubernetes.io/projected/2ff9a922-cd91-4600-988c-276b1a848b0e-kube-api-access-8ls4l\") pod \"kuadrant-operator-controller-manager-86878c769f-wkpq4\" (UID: \"2ff9a922-cd91-4600-988c-276b1a848b0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.532700 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.532670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff9a922-cd91-4600-988c-276b1a848b0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wkpq4\" (UID: \"2ff9a922-cd91-4600-988c-276b1a848b0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.541248 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.541229    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ls4l\" (UniqueName: \"kubernetes.io/projected/2ff9a922-cd91-4600-988c-276b1a848b0e-kube-api-access-8ls4l\") pod \"kuadrant-operator-controller-manager-86878c769f-wkpq4\" (UID: \"2ff9a922-cd91-4600-988c-276b1a848b0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.659229 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.659132    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:22.794538 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:22.794502    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"]
Jul 28 14:39:22.797933 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:39:22.797899    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff9a922_cd91_4600_988c_276b1a848b0e.slice/crio-3faffae9fb5bb47d7fe92cc439e212610afd497088bda14e1363e5fe6cfb5d7d WatchSource:0}: Error finding container 3faffae9fb5bb47d7fe92cc439e212610afd497088bda14e1363e5fe6cfb5d7d: Status 404 returned error can't find the container with id 3faffae9fb5bb47d7fe92cc439e212610afd497088bda14e1363e5fe6cfb5d7d
Jul 28 14:39:23.163312 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:23.163278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4" event={"ID":"2ff9a922-cd91-4600-988c-276b1a848b0e","Type":"ContainerStarted","Data":"35e821290b1e09380bdf8163483e1fcf2e566cc4a4423b14aa98cfdc93573287"}
Jul 28 14:39:23.163479 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:23.163371    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:39:23.163479 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:23.163390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4" event={"ID":"2ff9a922-cd91-4600-988c-276b1a848b0e","Type":"ContainerStarted","Data":"3faffae9fb5bb47d7fe92cc439e212610afd497088bda14e1363e5fe6cfb5d7d"}
Jul 28 14:39:23.187378 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:23.187332    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4" podStartSLOduration=1.187317895 podStartE2EDuration="1.187317895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:39:23.185037726 +0000 UTC m=+1233.472898669" watchObservedRunningTime="2026-07-28 14:39:23.187317895 +0000 UTC m=+1233.475178893"
Jul 28 14:39:34.172843 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:39:34.172814    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wkpq4"
Jul 28 14:42:12.462170 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.462134    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"]
Jul 28 14:42:12.465795 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.465773    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.468706 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.468681    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-34e4f271-kserve-self-signed-certs\""
Jul 28 14:42:12.482144 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.482122    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"]
Jul 28 14:42:12.565313 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.565288    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-home\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.565313 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.565315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-model-cache\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.565553 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.565364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tls-certs\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.565553 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.565463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tmp-dir\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.565553 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.565526    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-dshm\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.565687 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.565555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spm44\" (UniqueName: \"kubernetes.io/projected/d4634404-b83a-4c3c-97b0-c33e0f66b44d-kube-api-access-spm44\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666136 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666109    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-dshm\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666136 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spm44\" (UniqueName: \"kubernetes.io/projected/d4634404-b83a-4c3c-97b0-c33e0f66b44d-kube-api-access-spm44\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666359 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-home\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666359 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-model-cache\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666359 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tls-certs\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666359 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tmp-dir\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666598 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-home\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666689 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666666    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-model-cache\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.666738 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.666678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tmp-dir\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.668507 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.668348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-dshm\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.672254 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.668974    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tls-certs\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.677064 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.677041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spm44\" (UniqueName: \"kubernetes.io/projected/d4634404-b83a-4c3c-97b0-c33e0f66b44d-kube-api-access-spm44\") pod \"e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") " pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.775675 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.775606    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:12.905613 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.905589    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"]
Jul 28 14:42:12.907516 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:42:12.907487    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4634404_b83a_4c3c_97b0_c33e0f66b44d.slice/crio-2971eedf35272c236ad3ca90e091a3db0925d833d4f991fc2164e986428b9523 WatchSource:0}: Error finding container 2971eedf35272c236ad3ca90e091a3db0925d833d4f991fc2164e986428b9523: Status 404 returned error can't find the container with id 2971eedf35272c236ad3ca90e091a3db0925d833d4f991fc2164e986428b9523
Jul 28 14:42:12.968673 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:12.968640    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9" event={"ID":"d4634404-b83a-4c3c-97b0-c33e0f66b44d","Type":"ContainerStarted","Data":"2971eedf35272c236ad3ca90e091a3db0925d833d4f991fc2164e986428b9523"}
Jul 28 14:42:13.975143 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:13.975111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9" event={"ID":"d4634404-b83a-4c3c-97b0-c33e0f66b44d","Type":"ContainerStarted","Data":"6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff"}
Jul 28 14:42:13.994161 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:13.994115    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9" podStartSLOduration=1.994099928 podStartE2EDuration="1.994099928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:42:13.992649331 +0000 UTC m=+1404.280510289" watchObservedRunningTime="2026-07-28 14:42:13.994099928 +0000 UTC m=+1404.281960871"
Jul 28 14:42:22.050851 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.050817    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"]
Jul 28 14:42:22.054222 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.054196    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.056980 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.056955    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-34e4f271-kserve-self-signed-certs\""
Jul 28 14:42:22.063040 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.063019    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"]
Jul 28 14:42:22.155622 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.155590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-dshm\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.155783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.155634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tls-certs\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.155783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.155709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tmp-dir\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.155783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.155739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-model-cache\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.155783 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.155761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-home\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.155922 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.155856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwdt5\" (UniqueName: \"kubernetes.io/projected/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-kube-api-access-rwdt5\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.256496 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.256462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-dshm\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.256676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.256507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tls-certs\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.256676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.256559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tmp-dir\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.256676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.256579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-model-cache\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.256676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.256601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-home\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.256676 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.256641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwdt5\" (UniqueName: \"kubernetes.io/projected/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-kube-api-access-rwdt5\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.257017 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.256988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-home\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.257099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.257020    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tmp-dir\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.257099 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.257048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-model-cache\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.258970 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.258948    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-dshm\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.259096 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.259079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tls-certs\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.264828 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.264805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwdt5\" (UniqueName: \"kubernetes.io/projected/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-kube-api-access-rwdt5\") pod \"e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") " pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.365271 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.365159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:22.492730 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.492706    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"]
Jul 28 14:42:22.494430 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:42:22.494402    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9124cb81_5efb_4cd6_89a3_982f7fa00e5f.slice/crio-1aceca4f81909de59f6e6340ad33e8bb298e745292feddfd5dd30ca41a8aa9da WatchSource:0}: Error finding container 1aceca4f81909de59f6e6340ad33e8bb298e745292feddfd5dd30ca41a8aa9da: Status 404 returned error can't find the container with id 1aceca4f81909de59f6e6340ad33e8bb298e745292feddfd5dd30ca41a8aa9da
Jul 28 14:42:22.776172 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.776141    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:22.776172 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.776181    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:22.788332 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:22.788308    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:23.019296 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:23.019257    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4" event={"ID":"9124cb81-5efb-4cd6-89a3-982f7fa00e5f","Type":"ContainerStarted","Data":"3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192"}
Jul 28 14:42:23.019512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:23.019304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4" event={"ID":"9124cb81-5efb-4cd6-89a3-982f7fa00e5f","Type":"ContainerStarted","Data":"1aceca4f81909de59f6e6340ad33e8bb298e745292feddfd5dd30ca41a8aa9da"}
Jul 28 14:42:23.032086 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:23.032012    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:42:23.039903 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:23.039856    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4" podStartSLOduration=1.039841603 podStartE2EDuration="1.039841603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:42:23.036828505 +0000 UTC m=+1413.324689447" watchObservedRunningTime="2026-07-28 14:42:23.039841603 +0000 UTC m=+1413.327702546"
Jul 28 14:42:32.365409 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:32.365371    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:32.365869 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:32.365426    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:32.377654 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:32.377630    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:33.082578 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:33.082548    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:42:55.761512 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:55.761432    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"]
Jul 28 14:42:57.251811 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:42:57.251774    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"]
Jul 28 14:43:10.780753 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:10.780700    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9" podUID="d4634404-b83a-4c3c-97b0-c33e0f66b44d" containerName="main" containerID="cri-o://6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff" gracePeriod=45
Jul 28 14:43:11.026711 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.026688    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:43:11.113256 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113143    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-dshm\") pod \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") "
Jul 28 14:43:11.113256 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113205    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tls-certs\") pod \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") "
Jul 28 14:43:11.113478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113282    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-model-cache\") pod \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") "
Jul 28 14:43:11.113478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113298    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tmp-dir\") pod \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") "
Jul 28 14:43:11.113478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113366    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-home\") pod \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") "
Jul 28 14:43:11.113478 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113396    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spm44\" (UniqueName: \"kubernetes.io/projected/d4634404-b83a-4c3c-97b0-c33e0f66b44d-kube-api-access-spm44\") pod \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\" (UID: \"d4634404-b83a-4c3c-97b0-c33e0f66b44d\") "
Jul 28 14:43:11.113694 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113582    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d4634404-b83a-4c3c-97b0-c33e0f66b44d" (UID: "d4634404-b83a-4c3c-97b0-c33e0f66b44d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:11.113694 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113630    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-home" (OuterVolumeSpecName: "home") pod "d4634404-b83a-4c3c-97b0-c33e0f66b44d" (UID: "d4634404-b83a-4c3c-97b0-c33e0f66b44d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:11.113694 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113637    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d4634404-b83a-4c3c-97b0-c33e0f66b44d" (UID: "d4634404-b83a-4c3c-97b0-c33e0f66b44d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:11.113845 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113728    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:11.113845 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113747    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:11.113845 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.113760    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:11.115466 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.115436    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-dshm" (OuterVolumeSpecName: "dshm") pod "d4634404-b83a-4c3c-97b0-c33e0f66b44d" (UID: "d4634404-b83a-4c3c-97b0-c33e0f66b44d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:11.115578 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.115470    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4634404-b83a-4c3c-97b0-c33e0f66b44d" (UID: "d4634404-b83a-4c3c-97b0-c33e0f66b44d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:43:11.115578 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.115499    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4634404-b83a-4c3c-97b0-c33e0f66b44d-kube-api-access-spm44" (OuterVolumeSpecName: "kube-api-access-spm44") pod "d4634404-b83a-4c3c-97b0-c33e0f66b44d" (UID: "d4634404-b83a-4c3c-97b0-c33e0f66b44d"). InnerVolumeSpecName "kube-api-access-spm44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:43:11.215096 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.215066    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4634404-b83a-4c3c-97b0-c33e0f66b44d-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:11.215096 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.215090    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4634404-b83a-4c3c-97b0-c33e0f66b44d-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:11.215096 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.215100    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spm44\" (UniqueName: \"kubernetes.io/projected/d4634404-b83a-4c3c-97b0-c33e0f66b44d-kube-api-access-spm44\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:11.254286 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.254254    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4634404-b83a-4c3c-97b0-c33e0f66b44d" containerID="6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff" exitCode=0
Jul 28 14:43:11.254454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.254298    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"
Jul 28 14:43:11.254454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.254342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9" event={"ID":"d4634404-b83a-4c3c-97b0-c33e0f66b44d","Type":"ContainerDied","Data":"6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff"}
Jul 28 14:43:11.254454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.254386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9" event={"ID":"d4634404-b83a-4c3c-97b0-c33e0f66b44d","Type":"ContainerDied","Data":"2971eedf35272c236ad3ca90e091a3db0925d833d4f991fc2164e986428b9523"}
Jul 28 14:43:11.254454 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.254403    2579 scope.go:117] "RemoveContainer" containerID="6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff"
Jul 28 14:43:11.264183 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.264164    2579 scope.go:117] "RemoveContainer" containerID="6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff"
Jul 28 14:43:11.264532 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:43:11.264511    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff\": container with ID starting with 6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff not found: ID does not exist" containerID="6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff"
Jul 28 14:43:11.264604 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.264540    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff"} err="failed to get container status \"6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff\": rpc error: code = NotFound desc = could not find container \"6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff\": container with ID starting with 6cd8dce7d9cacbaf0ace386fb8a3ad12f39782906f9e3d99efd5a66c23dc30ff not found: ID does not exist"
Jul 28 14:43:11.275988 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.275966    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"]
Jul 28 14:43:11.279596 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:11.279576    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-34e4f271-kserve-bc786746c-t4cv9"]
Jul 28 14:43:12.192663 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.192630    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4634404-b83a-4c3c-97b0-c33e0f66b44d" path="/var/lib/kubelet/pods/d4634404-b83a-4c3c-97b0-c33e0f66b44d/volumes"
Jul 28 14:43:12.271108 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.271072    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4" podUID="9124cb81-5efb-4cd6-89a3-982f7fa00e5f" containerName="main" containerID="cri-o://3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192" gracePeriod=45
Jul 28 14:43:12.528933 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.528909    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:43:12.626020 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.625991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-model-cache\") pod \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") "
Jul 28 14:43:12.626195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626036    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-home\") pod \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") "
Jul 28 14:43:12.626195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626076    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tls-certs\") pod \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") "
Jul 28 14:43:12.626195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626119    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tmp-dir\") pod \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") "
Jul 28 14:43:12.626195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626134    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwdt5\" (UniqueName: \"kubernetes.io/projected/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-kube-api-access-rwdt5\") pod \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") "
Jul 28 14:43:12.626195 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-dshm\") pod \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\" (UID: \"9124cb81-5efb-4cd6-89a3-982f7fa00e5f\") "
Jul 28 14:43:12.626497 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626308    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "9124cb81-5efb-4cd6-89a3-982f7fa00e5f" (UID: "9124cb81-5efb-4cd6-89a3-982f7fa00e5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:12.626497 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626332    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-home" (OuterVolumeSpecName: "home") pod "9124cb81-5efb-4cd6-89a3-982f7fa00e5f" (UID: "9124cb81-5efb-4cd6-89a3-982f7fa00e5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:12.626497 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626443    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9124cb81-5efb-4cd6-89a3-982f7fa00e5f" (UID: "9124cb81-5efb-4cd6-89a3-982f7fa00e5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:12.626639 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626548    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:12.626639 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626564    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:12.626639 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.626578    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:12.628173 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.628146    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-dshm" (OuterVolumeSpecName: "dshm") pod "9124cb81-5efb-4cd6-89a3-982f7fa00e5f" (UID: "9124cb81-5efb-4cd6-89a3-982f7fa00e5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 14:43:12.628302 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.628194    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9124cb81-5efb-4cd6-89a3-982f7fa00e5f" (UID: "9124cb81-5efb-4cd6-89a3-982f7fa00e5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 14:43:12.628439 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.628420    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-kube-api-access-rwdt5" (OuterVolumeSpecName: "kube-api-access-rwdt5") pod "9124cb81-5efb-4cd6-89a3-982f7fa00e5f" (UID: "9124cb81-5efb-4cd6-89a3-982f7fa00e5f"). InnerVolumeSpecName "kube-api-access-rwdt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 14:43:12.727695 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.727660    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:12.727695 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.727692    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:12.727879 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:12.727708    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwdt5\" (UniqueName: \"kubernetes.io/projected/9124cb81-5efb-4cd6-89a3-982f7fa00e5f-kube-api-access-rwdt5\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 14:43:13.266132 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.266095    2579 generic.go:356] "Generic (PLEG): container finished" podID="9124cb81-5efb-4cd6-89a3-982f7fa00e5f" containerID="3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192" exitCode=0
Jul 28 14:43:13.266132 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.266139    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"
Jul 28 14:43:13.266679 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.266182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4" event={"ID":"9124cb81-5efb-4cd6-89a3-982f7fa00e5f","Type":"ContainerDied","Data":"3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192"}
Jul 28 14:43:13.266679 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.266244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4" event={"ID":"9124cb81-5efb-4cd6-89a3-982f7fa00e5f","Type":"ContainerDied","Data":"1aceca4f81909de59f6e6340ad33e8bb298e745292feddfd5dd30ca41a8aa9da"}
Jul 28 14:43:13.266679 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.266267    2579 scope.go:117] "RemoveContainer" containerID="3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192"
Jul 28 14:43:13.276301 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.276118    2579 scope.go:117] "RemoveContainer" containerID="3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192"
Jul 28 14:43:13.276384 ip-10-0-141-50 kubenswrapper[2579]: E0728 14:43:13.276367    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192\": container with ID starting with 3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192 not found: ID does not exist" containerID="3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192"
Jul 28 14:43:13.276426 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.276392    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192"} err="failed to get container status \"3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192\": rpc error: code = NotFound desc = could not find container \"3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192\": container with ID starting with 3140865598a292802a5315ccca33d6c03be41e61db2663ea7ce9f0abde43d192 not found: ID does not exist"
Jul 28 14:43:13.288329 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.288304    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"]
Jul 28 14:43:13.292539 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:13.292516    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-34e4f271-kserve-65d99f5bb7-9nlr4"]
Jul 28 14:43:14.192583 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:14.192553    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9124cb81-5efb-4cd6-89a3-982f7fa00e5f" path="/var/lib/kubelet/pods/9124cb81-5efb-4cd6-89a3-982f7fa00e5f/volumes"
Jul 28 14:43:50.152490 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:50.152454    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:43:50.160136 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:43:50.160118    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:48:50.181123 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:48:50.181095    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:48:50.190931 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:48:50.190904    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:53:50.210039 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:53:50.210007    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:53:50.221158 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:53:50.221138    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:58:50.240256 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:58:50.240135    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:58:50.253669 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:58:50.253649    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 14:59:43.655008 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.654971    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"]
Jul 28 14:59:43.655592 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.655576    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4634404-b83a-4c3c-97b0-c33e0f66b44d" containerName="main"
Jul 28 14:59:43.655638 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.655596    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4634404-b83a-4c3c-97b0-c33e0f66b44d" containerName="main"
Jul 28 14:59:43.655638 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.655624    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9124cb81-5efb-4cd6-89a3-982f7fa00e5f" containerName="main"
Jul 28 14:59:43.655638 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.655632    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9124cb81-5efb-4cd6-89a3-982f7fa00e5f" containerName="main"
Jul 28 14:59:43.655734 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.655721    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9124cb81-5efb-4cd6-89a3-982f7fa00e5f" containerName="main"
Jul 28 14:59:43.655768 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.655733    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4634404-b83a-4c3c-97b0-c33e0f66b44d" containerName="main"
Jul 28 14:59:43.659131 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.659114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.663714 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.663692    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-917bdcc4\"/\"default-dockercfg-7fxx7\""
Jul 28 14:59:43.663842 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.663753    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-917bdcc4\"/\"e2e-lifecycle-model-917bdcc4-kserve-self-signed-certs\""
Jul 28 14:59:43.663842 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.663770    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-917bdcc4\"/\"kube-root-ca.crt\""
Jul 28 14:59:43.663842 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.663755    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-917bdcc4\"/\"openshift-service-ca.crt\""
Jul 28 14:59:43.667153 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.667131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"]
Jul 28 14:59:43.686481 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.686456    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f6b7a-8c95-4470-838a-c5368da56ac0-tls-certs\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.686589 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.686493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-tmp-dir\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.686589 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.686567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-dshm\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.686675 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.686600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-model-cache\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.686675 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.686642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp5g2\" (UniqueName: \"kubernetes.io/projected/c90f6b7a-8c95-4470-838a-c5368da56ac0-kube-api-access-bp5g2\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.686745 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.686698    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-home\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.787752 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.787723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-tmp-dir\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.787880 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.787779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-dshm\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.787928 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.787907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-model-cache\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.788006 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.787988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp5g2\" (UniqueName: \"kubernetes.io/projected/c90f6b7a-8c95-4470-838a-c5368da56ac0-kube-api-access-bp5g2\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.788058 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.788041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-home\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.788117 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.788080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f6b7a-8c95-4470-838a-c5368da56ac0-tls-certs\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.788172 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.788148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-tmp-dir\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.788411 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.788283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-model-cache\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.788411 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.788381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-home\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.790033 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.790015    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-dshm\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.790426 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.790411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f6b7a-8c95-4470-838a-c5368da56ac0-tls-certs\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.797638 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.797617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp5g2\" (UniqueName: \"kubernetes.io/projected/c90f6b7a-8c95-4470-838a-c5368da56ac0-kube-api-access-bp5g2\") pod \"e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") " pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:43.968953 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:43.968918    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:44.099721 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:44.099696    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"]
Jul 28 14:59:44.101893 ip-10-0-141-50 kubenswrapper[2579]: W0728 14:59:44.101867    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc90f6b7a_8c95_4470_838a_c5368da56ac0.slice/crio-7f2c843915cfeb22357372716fd9473ac5918675ccdf9c071ae95b18f4751e4e WatchSource:0}: Error finding container 7f2c843915cfeb22357372716fd9473ac5918675ccdf9c071ae95b18f4751e4e: Status 404 returned error can't find the container with id 7f2c843915cfeb22357372716fd9473ac5918675ccdf9c071ae95b18f4751e4e
Jul 28 14:59:44.997550 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:44.997513    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8" event={"ID":"c90f6b7a-8c95-4470-838a-c5368da56ac0","Type":"ContainerStarted","Data":"745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994"}
Jul 28 14:59:44.997550 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:44.997556    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8" event={"ID":"c90f6b7a-8c95-4470-838a-c5368da56ac0","Type":"ContainerStarted","Data":"7f2c843915cfeb22357372716fd9473ac5918675ccdf9c071ae95b18f4751e4e"}
Jul 28 14:59:45.016523 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:45.016473    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8" podStartSLOduration=2.016457438 podStartE2EDuration="2.016457438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:59:45.015911592 +0000 UTC m=+2455.303772534" watchObservedRunningTime="2026-07-28 14:59:45.016457438 +0000 UTC m=+2455.304318379"
Jul 28 14:59:53.970025 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:53.969978    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:53.970025 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:53.970034    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:53.982232 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:53.982198    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 14:59:54.048409 ip-10-0-141-50 kubenswrapper[2579]: I0728 14:59:54.048385    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 15:00:23.380288 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:23.380255    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"]
Jul 28 15:00:38.399073 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.399011    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8" podUID="c90f6b7a-8c95-4470-838a-c5368da56ac0" containerName="main" containerID="cri-o://745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994" gracePeriod=45
Jul 28 15:00:38.646906 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.646874    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 15:00:38.789168 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789124    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-dshm\") pod \"c90f6b7a-8c95-4470-838a-c5368da56ac0\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") "
Jul 28 15:00:38.789380 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789189    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp5g2\" (UniqueName: \"kubernetes.io/projected/c90f6b7a-8c95-4470-838a-c5368da56ac0-kube-api-access-bp5g2\") pod \"c90f6b7a-8c95-4470-838a-c5368da56ac0\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") "
Jul 28 15:00:38.789380 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789234    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-tmp-dir\") pod \"c90f6b7a-8c95-4470-838a-c5368da56ac0\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") "
Jul 28 15:00:38.789380 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f6b7a-8c95-4470-838a-c5368da56ac0-tls-certs\") pod \"c90f6b7a-8c95-4470-838a-c5368da56ac0\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") "
Jul 28 15:00:38.789380 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789343    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-model-cache\") pod \"c90f6b7a-8c95-4470-838a-c5368da56ac0\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") "
Jul 28 15:00:38.789643 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789383    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-home\") pod \"c90f6b7a-8c95-4470-838a-c5368da56ac0\" (UID: \"c90f6b7a-8c95-4470-838a-c5368da56ac0\") "
Jul 28 15:00:38.789643 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789538    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c90f6b7a-8c95-4470-838a-c5368da56ac0" (UID: "c90f6b7a-8c95-4470-838a-c5368da56ac0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:00:38.789643 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789573    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-model-cache" (OuterVolumeSpecName: "model-cache") pod "c90f6b7a-8c95-4470-838a-c5368da56ac0" (UID: "c90f6b7a-8c95-4470-838a-c5368da56ac0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:00:38.789807 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789691    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-home" (OuterVolumeSpecName: "home") pod "c90f6b7a-8c95-4470-838a-c5368da56ac0" (UID: "c90f6b7a-8c95-4470-838a-c5368da56ac0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:00:38.789866 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789830    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:00:38.789866 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789842    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:00:38.789866 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.789851    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:00:38.791492 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.791468    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-dshm" (OuterVolumeSpecName: "dshm") pod "c90f6b7a-8c95-4470-838a-c5368da56ac0" (UID: "c90f6b7a-8c95-4470-838a-c5368da56ac0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:00:38.791595 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.791502    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c90f6b7a-8c95-4470-838a-c5368da56ac0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c90f6b7a-8c95-4470-838a-c5368da56ac0" (UID: "c90f6b7a-8c95-4470-838a-c5368da56ac0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:00:38.791595 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.791513    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c90f6b7a-8c95-4470-838a-c5368da56ac0-kube-api-access-bp5g2" (OuterVolumeSpecName: "kube-api-access-bp5g2") pod "c90f6b7a-8c95-4470-838a-c5368da56ac0" (UID: "c90f6b7a-8c95-4470-838a-c5368da56ac0"). InnerVolumeSpecName "kube-api-access-bp5g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:00:38.890316 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.890276    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f6b7a-8c95-4470-838a-c5368da56ac0-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:00:38.890316 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.890304    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f6b7a-8c95-4470-838a-c5368da56ac0-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:00:38.890316 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:38.890314    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp5g2\" (UniqueName: \"kubernetes.io/projected/c90f6b7a-8c95-4470-838a-c5368da56ac0-kube-api-access-bp5g2\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:00:39.248440 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.248401    2579 generic.go:356] "Generic (PLEG): container finished" podID="c90f6b7a-8c95-4470-838a-c5368da56ac0" containerID="745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994" exitCode=0
Jul 28 15:00:39.248672 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.248449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"
Jul 28 15:00:39.248672 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.248476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8" event={"ID":"c90f6b7a-8c95-4470-838a-c5368da56ac0","Type":"ContainerDied","Data":"745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994"}
Jul 28 15:00:39.248672 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.248512    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8" event={"ID":"c90f6b7a-8c95-4470-838a-c5368da56ac0","Type":"ContainerDied","Data":"7f2c843915cfeb22357372716fd9473ac5918675ccdf9c071ae95b18f4751e4e"}
Jul 28 15:00:39.248672 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.248528    2579 scope.go:117] "RemoveContainer" containerID="745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994"
Jul 28 15:00:39.257339 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.257317    2579 scope.go:117] "RemoveContainer" containerID="745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994"
Jul 28 15:00:39.257558 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:00:39.257539    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994\": container with ID starting with 745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994 not found: ID does not exist" containerID="745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994"
Jul 28 15:00:39.257619 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.257565    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994"} err="failed to get container status \"745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994\": rpc error: code = NotFound desc = could not find container \"745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994\": container with ID starting with 745ae0ce524edde3bba8e33f620fd410ca9aaa089df17083df2962828dbac994 not found: ID does not exist"
Jul 28 15:00:39.270575 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.270552    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"]
Jul 28 15:00:39.275044 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:39.275024    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-917bdcc4/e2e-lifecycle-model-917bdcc4-kserve-7fd5b7bfb7-n2ql8"]
Jul 28 15:00:40.192461 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:00:40.192430    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c90f6b7a-8c95-4470-838a-c5368da56ac0" path="/var/lib/kubelet/pods/c90f6b7a-8c95-4470-838a-c5368da56ac0/volumes"
Jul 28 15:03:50.269831 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:03:50.269727    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:03:50.284075 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:03:50.284055    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:08:33.168795 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.168758    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"]
Jul 28 15:08:33.169283 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.169152    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90f6b7a-8c95-4470-838a-c5368da56ac0" containerName="main"
Jul 28 15:08:33.169283 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.169163    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90f6b7a-8c95-4470-838a-c5368da56ac0" containerName="main"
Jul 28 15:08:33.169283 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.169263    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90f6b7a-8c95-4470-838a-c5368da56ac0" containerName="main"
Jul 28 15:08:33.172420 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.172403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.177134 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.177097    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-06cd69\"/\"default-dockercfg-55l2t\""
Jul 28 15:08:33.177331 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.177157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-06cd69\"/\"kube-root-ca.crt\""
Jul 28 15:08:33.177331 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.177163    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-06cd69\"/\"openshift-service-ca.crt\""
Jul 28 15:08:33.177331 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.177157    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-06cd69\"/\"test-model-06cd69-kserve-self-signed-certs\""
Jul 28 15:08:33.182849 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.182829    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"]
Jul 28 15:08:33.255802 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.255771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh7g6\" (UniqueName: \"kubernetes.io/projected/7859165f-c145-4b0f-a7b6-c746f50357d9-kube-api-access-kh7g6\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.255955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.255837    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-tmp-dir\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.255955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.255854    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-home\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.255955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.255872    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7859165f-c145-4b0f-a7b6-c746f50357d9-tls-certs\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.255955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.255891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-dshm\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.255955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.255917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-model-cache\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357004 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.356981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7859165f-c145-4b0f-a7b6-c746f50357d9-tls-certs\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357143 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-dshm\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357143 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-model-cache\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357143 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh7g6\" (UniqueName: \"kubernetes.io/projected/7859165f-c145-4b0f-a7b6-c746f50357d9-kube-api-access-kh7g6\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357143 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-tmp-dir\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357143 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-home\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357508 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-model-cache\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357559 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-tmp-dir\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.357559 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.357546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-home\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.359258 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.359240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-dshm\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.359561 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.359546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7859165f-c145-4b0f-a7b6-c746f50357d9-tls-certs\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.366114 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.366092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh7g6\" (UniqueName: \"kubernetes.io/projected/7859165f-c145-4b0f-a7b6-c746f50357d9-kube-api-access-kh7g6\") pod \"test-model-06cd69-kserve-ccb58545-8885g\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") " pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.481629 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.481603    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:33.612506 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:33.612398    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"]
Jul 28 15:08:33.615046 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:08:33.615021    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7859165f_c145_4b0f_a7b6_c746f50357d9.slice/crio-f62ce24659d38cb136ce9c9dcbed5b8e2b8df8c82c1e0d3c52454fb5b307e1ba WatchSource:0}: Error finding container f62ce24659d38cb136ce9c9dcbed5b8e2b8df8c82c1e0d3c52454fb5b307e1ba: Status 404 returned error can't find the container with id f62ce24659d38cb136ce9c9dcbed5b8e2b8df8c82c1e0d3c52454fb5b307e1ba
Jul 28 15:08:34.443738 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:34.443705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g" event={"ID":"7859165f-c145-4b0f-a7b6-c746f50357d9","Type":"ContainerStarted","Data":"976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a"}
Jul 28 15:08:34.444107 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:34.443753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g" event={"ID":"7859165f-c145-4b0f-a7b6-c746f50357d9","Type":"ContainerStarted","Data":"f62ce24659d38cb136ce9c9dcbed5b8e2b8df8c82c1e0d3c52454fb5b307e1ba"}
Jul 28 15:08:34.465384 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:34.465338    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g" podStartSLOduration=1.465323736 podStartE2EDuration="1.465323736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:08:34.46267234 +0000 UTC m=+2984.750533310" watchObservedRunningTime="2026-07-28 15:08:34.465323736 +0000 UTC m=+2984.753184680"
Jul 28 15:08:43.482779 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:43.482739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:43.483313 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:43.482797    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:43.499748 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:43.499725    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:43.511065 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:43.511045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:08:50.307402 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:50.307303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:08:50.315027 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:08:50.315008    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:09:18.210817 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.210774    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"]
Jul 28 15:09:18.213859 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.213841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.216779 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.216758    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f4853e\"/\"test-model-f4853e-kserve-self-signed-certs\""
Jul 28 15:09:18.218141 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.218113    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f4853e\"/\"openshift-service-ca.crt\""
Jul 28 15:09:18.218141 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.218127    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f4853e\"/\"kube-root-ca.crt\""
Jul 28 15:09:18.218317 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.218303    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f4853e\"/\"default-dockercfg-s74kh\""
Jul 28 15:09:18.244364 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.244344    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"]
Jul 28 15:09:18.261612 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.261593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-home\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.261713 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.261618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46kwh\" (UniqueName: \"kubernetes.io/projected/bd8a5871-929a-454e-95fe-4d339bee998b-kube-api-access-46kwh\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.261713 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.261656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-dshm\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.261829 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.261717    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-tmp-dir\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.261829 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.261800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-model-cache\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.261947 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.261855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8a5871-929a-454e-95fe-4d339bee998b-tls-certs\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363289 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-home\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363289 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46kwh\" (UniqueName: \"kubernetes.io/projected/bd8a5871-929a-454e-95fe-4d339bee998b-kube-api-access-46kwh\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363468 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-dshm\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363468 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-tmp-dir\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363468 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-model-cache\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363468 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8a5871-929a-454e-95fe-4d339bee998b-tls-certs\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363732 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-home\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363801 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363761    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-tmp-dir\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.363854 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.363798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-model-cache\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.365529 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.365513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-dshm\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.365814 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.365799    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8a5871-929a-454e-95fe-4d339bee998b-tls-certs\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.375307 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.375279    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46kwh\" (UniqueName: \"kubernetes.io/projected/bd8a5871-929a-454e-95fe-4d339bee998b-kube-api-access-46kwh\") pod \"test-model-f4853e-kserve-58d4fd8cd7-zwvlm\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") " pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.522980 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.522899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:18.667061 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:18.667037    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"]
Jul 28 15:09:18.668475 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:09:18.668454    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd8a5871_929a_454e_95fe_4d339bee998b.slice/crio-31d0ffcf5585d9671c90a66db48a8ac3b2b970397c74d031ff83f6d99b5ab23b WatchSource:0}: Error finding container 31d0ffcf5585d9671c90a66db48a8ac3b2b970397c74d031ff83f6d99b5ab23b: Status 404 returned error can't find the container with id 31d0ffcf5585d9671c90a66db48a8ac3b2b970397c74d031ff83f6d99b5ab23b
Jul 28 15:09:19.661858 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:19.661824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm" event={"ID":"bd8a5871-929a-454e-95fe-4d339bee998b","Type":"ContainerStarted","Data":"2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384"}
Jul 28 15:09:19.661858 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:19.661864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm" event={"ID":"bd8a5871-929a-454e-95fe-4d339bee998b","Type":"ContainerStarted","Data":"31d0ffcf5585d9671c90a66db48a8ac3b2b970397c74d031ff83f6d99b5ab23b"}
Jul 28 15:09:19.684576 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:19.684530    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm" podStartSLOduration=1.68451492 podStartE2EDuration="1.68451492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:09:19.68305665 +0000 UTC m=+3029.970917621" watchObservedRunningTime="2026-07-28 15:09:19.68451492 +0000 UTC m=+3029.972375863"
Jul 28 15:09:28.523775 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:28.523746    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:28.524122 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:28.523792    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:28.536067 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:28.536044    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:09:28.712818 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:09:28.712790    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:10:22.272645 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:22.272570    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"]
Jul 28 15:10:23.192288 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:23.192259    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"]
Jul 28 15:10:37.291226 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.291171    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g" podUID="7859165f-c145-4b0f-a7b6-c746f50357d9" containerName="main" containerID="cri-o://976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a" gracePeriod=45
Jul 28 15:10:37.534604 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.534575    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:10:37.579462 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579395    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7859165f-c145-4b0f-a7b6-c746f50357d9-tls-certs\") pod \"7859165f-c145-4b0f-a7b6-c746f50357d9\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") "
Jul 28 15:10:37.579462 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579430    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh7g6\" (UniqueName: \"kubernetes.io/projected/7859165f-c145-4b0f-a7b6-c746f50357d9-kube-api-access-kh7g6\") pod \"7859165f-c145-4b0f-a7b6-c746f50357d9\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") "
Jul 28 15:10:37.579654 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579471    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-dshm\") pod \"7859165f-c145-4b0f-a7b6-c746f50357d9\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") "
Jul 28 15:10:37.579720 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579652    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-model-cache\") pod \"7859165f-c145-4b0f-a7b6-c746f50357d9\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") "
Jul 28 15:10:37.579775 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579725    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-home\") pod \"7859165f-c145-4b0f-a7b6-c746f50357d9\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") "
Jul 28 15:10:37.579826 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-tmp-dir\") pod \"7859165f-c145-4b0f-a7b6-c746f50357d9\" (UID: \"7859165f-c145-4b0f-a7b6-c746f50357d9\") "
Jul 28 15:10:37.580003 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579959    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "7859165f-c145-4b0f-a7b6-c746f50357d9" (UID: "7859165f-c145-4b0f-a7b6-c746f50357d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:37.580003 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.579969    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-home" (OuterVolumeSpecName: "home") pod "7859165f-c145-4b0f-a7b6-c746f50357d9" (UID: "7859165f-c145-4b0f-a7b6-c746f50357d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:37.580122 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.580077    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7859165f-c145-4b0f-a7b6-c746f50357d9" (UID: "7859165f-c145-4b0f-a7b6-c746f50357d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:37.580202 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.580187    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:37.580272 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.580208    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:37.580272 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.580246    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:37.581757 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.581728    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7859165f-c145-4b0f-a7b6-c746f50357d9-kube-api-access-kh7g6" (OuterVolumeSpecName: "kube-api-access-kh7g6") pod "7859165f-c145-4b0f-a7b6-c746f50357d9" (UID: "7859165f-c145-4b0f-a7b6-c746f50357d9"). InnerVolumeSpecName "kube-api-access-kh7g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:37.582010 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.581989    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7859165f-c145-4b0f-a7b6-c746f50357d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7859165f-c145-4b0f-a7b6-c746f50357d9" (UID: "7859165f-c145-4b0f-a7b6-c746f50357d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:10:37.582087 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.582054    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-dshm" (OuterVolumeSpecName: "dshm") pod "7859165f-c145-4b0f-a7b6-c746f50357d9" (UID: "7859165f-c145-4b0f-a7b6-c746f50357d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:37.681005 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.680958    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7859165f-c145-4b0f-a7b6-c746f50357d9-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:37.681005 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.680998    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh7g6\" (UniqueName: \"kubernetes.io/projected/7859165f-c145-4b0f-a7b6-c746f50357d9-kube-api-access-kh7g6\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:37.681005 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:37.681008    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7859165f-c145-4b0f-a7b6-c746f50357d9-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:38.040583 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.040548    2579 generic.go:356] "Generic (PLEG): container finished" podID="7859165f-c145-4b0f-a7b6-c746f50357d9" containerID="976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a" exitCode=0
Jul 28 15:10:38.040753 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.040590    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"
Jul 28 15:10:38.040753 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.040635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g" event={"ID":"7859165f-c145-4b0f-a7b6-c746f50357d9","Type":"ContainerDied","Data":"976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a"}
Jul 28 15:10:38.040753 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.040680    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g" event={"ID":"7859165f-c145-4b0f-a7b6-c746f50357d9","Type":"ContainerDied","Data":"f62ce24659d38cb136ce9c9dcbed5b8e2b8df8c82c1e0d3c52454fb5b307e1ba"}
Jul 28 15:10:38.040753 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.040701    2579 scope.go:117] "RemoveContainer" containerID="976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a"
Jul 28 15:10:38.049668 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.049652    2579 scope.go:117] "RemoveContainer" containerID="976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a"
Jul 28 15:10:38.049927 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:10:38.049908    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a\": container with ID starting with 976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a not found: ID does not exist" containerID="976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a"
Jul 28 15:10:38.049999 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.049936    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a"} err="failed to get container status \"976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a\": rpc error: code = NotFound desc = could not find container \"976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a\": container with ID starting with 976b10216e32b97b562804966780827e1341ab71c3b35f0b37d456232421944a not found: ID does not exist"
Jul 28 15:10:38.075072 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.075044    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"]
Jul 28 15:10:38.081505 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.081481    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-06cd69/test-model-06cd69-kserve-ccb58545-8885g"]
Jul 28 15:10:38.193013 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.192979    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7859165f-c145-4b0f-a7b6-c746f50357d9" path="/var/lib/kubelet/pods/7859165f-c145-4b0f-a7b6-c746f50357d9/volumes"
Jul 28 15:10:38.210903 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.210862    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm" podUID="bd8a5871-929a-454e-95fe-4d339bee998b" containerName="main" containerID="cri-o://2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384" gracePeriod=45
Jul 28 15:10:38.458639 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.458615    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:10:38.486747 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.486724    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8a5871-929a-454e-95fe-4d339bee998b-tls-certs\") pod \"bd8a5871-929a-454e-95fe-4d339bee998b\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") "
Jul 28 15:10:38.486886 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.486783    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-dshm\") pod \"bd8a5871-929a-454e-95fe-4d339bee998b\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") "
Jul 28 15:10:38.486886 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.486818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-tmp-dir\") pod \"bd8a5871-929a-454e-95fe-4d339bee998b\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") "
Jul 28 15:10:38.486886 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.486851    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-model-cache\") pod \"bd8a5871-929a-454e-95fe-4d339bee998b\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") "
Jul 28 15:10:38.486886 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.486871    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-home\") pod \"bd8a5871-929a-454e-95fe-4d339bee998b\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") "
Jul 28 15:10:38.487106 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.486894    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46kwh\" (UniqueName: \"kubernetes.io/projected/bd8a5871-929a-454e-95fe-4d339bee998b-kube-api-access-46kwh\") pod \"bd8a5871-929a-454e-95fe-4d339bee998b\" (UID: \"bd8a5871-929a-454e-95fe-4d339bee998b\") "
Jul 28 15:10:38.487167 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.487103    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd8a5871-929a-454e-95fe-4d339bee998b" (UID: "bd8a5871-929a-454e-95fe-4d339bee998b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:38.487246 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.487169    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-home" (OuterVolumeSpecName: "home") pod "bd8a5871-929a-454e-95fe-4d339bee998b" (UID: "bd8a5871-929a-454e-95fe-4d339bee998b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:38.487302 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.487255    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:38.487374 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.487306    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd8a5871-929a-454e-95fe-4d339bee998b" (UID: "bd8a5871-929a-454e-95fe-4d339bee998b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:38.488968 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.488949    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8a5871-929a-454e-95fe-4d339bee998b-kube-api-access-46kwh" (OuterVolumeSpecName: "kube-api-access-46kwh") pod "bd8a5871-929a-454e-95fe-4d339bee998b" (UID: "bd8a5871-929a-454e-95fe-4d339bee998b"). InnerVolumeSpecName "kube-api-access-46kwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:10:38.489172 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.489149    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-dshm" (OuterVolumeSpecName: "dshm") pod "bd8a5871-929a-454e-95fe-4d339bee998b" (UID: "bd8a5871-929a-454e-95fe-4d339bee998b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:10:38.489172 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.489157    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8a5871-929a-454e-95fe-4d339bee998b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd8a5871-929a-454e-95fe-4d339bee998b" (UID: "bd8a5871-929a-454e-95fe-4d339bee998b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:10:38.588621 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.588546    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:38.588621 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.588570    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:38.588621 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.588580    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46kwh\" (UniqueName: \"kubernetes.io/projected/bd8a5871-929a-454e-95fe-4d339bee998b-kube-api-access-46kwh\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:38.588621 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.588590    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd8a5871-929a-454e-95fe-4d339bee998b-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:38.588621 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:38.588604    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd8a5871-929a-454e-95fe-4d339bee998b-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:10:39.047807 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.047773    2579 generic.go:356] "Generic (PLEG): container finished" podID="bd8a5871-929a-454e-95fe-4d339bee998b" containerID="2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384" exitCode=0
Jul 28 15:10:39.047807 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.047807    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"
Jul 28 15:10:39.048015 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.047854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm" event={"ID":"bd8a5871-929a-454e-95fe-4d339bee998b","Type":"ContainerDied","Data":"2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384"}
Jul 28 15:10:39.048015 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.047892    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm" event={"ID":"bd8a5871-929a-454e-95fe-4d339bee998b","Type":"ContainerDied","Data":"31d0ffcf5585d9671c90a66db48a8ac3b2b970397c74d031ff83f6d99b5ab23b"}
Jul 28 15:10:39.048015 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.047910    2579 scope.go:117] "RemoveContainer" containerID="2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384"
Jul 28 15:10:39.059374 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.059349    2579 scope.go:117] "RemoveContainer" containerID="2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384"
Jul 28 15:10:39.059643 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:10:39.059621    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384\": container with ID starting with 2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384 not found: ID does not exist" containerID="2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384"
Jul 28 15:10:39.059711 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.059651    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384"} err="failed to get container status \"2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384\": rpc error: code = NotFound desc = could not find container \"2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384\": container with ID starting with 2cf08485b07a0f3345b9f66a1aa89159cfad54c3eb7fa4164b1ba042abae8384 not found: ID does not exist"
Jul 28 15:10:39.077612 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.077587    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"]
Jul 28 15:10:39.087009 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:39.086991    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f4853e/test-model-f4853e-kserve-58d4fd8cd7-zwvlm"]
Jul 28 15:10:40.198968 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:10:40.198930    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8a5871-929a-454e-95fe-4d339bee998b" path="/var/lib/kubelet/pods/bd8a5871-929a-454e-95fe-4d339bee998b/volumes"
Jul 28 15:13:50.337477 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:50.337362    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:13:50.345164 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:50.345145    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:13:59.066422 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.066389    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"]
Jul 28 15:13:59.066890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.066781    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8a5871-929a-454e-95fe-4d339bee998b" containerName="main"
Jul 28 15:13:59.066890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.066793    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8a5871-929a-454e-95fe-4d339bee998b" containerName="main"
Jul 28 15:13:59.066890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.066813    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7859165f-c145-4b0f-a7b6-c746f50357d9" containerName="main"
Jul 28 15:13:59.066890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.066819    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7859165f-c145-4b0f-a7b6-c746f50357d9" containerName="main"
Jul 28 15:13:59.066890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.066890    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7859165f-c145-4b0f-a7b6-c746f50357d9" containerName="main"
Jul 28 15:13:59.067166 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.066898    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8a5871-929a-454e-95fe-4d339bee998b" containerName="main"
Jul 28 15:13:59.069887 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.069870    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.074088 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.074060    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"default-dockercfg-sqhrp\""
Jul 28 15:13:59.074195 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.074090    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"auth-test-model-d96a19-kserve-self-signed-certs\""
Jul 28 15:13:59.074195 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.074068    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"kube-root-ca.crt\""
Jul 28 15:13:59.074195 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.074060    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"openshift-service-ca.crt\""
Jul 28 15:13:59.086825 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.086804    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"]
Jul 28 15:13:59.208924 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.208888    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-model-cache\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.208924 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.208930    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-home\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.209154 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.208976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/0c3ccdb5-fd8f-401b-b831-f17bda5db162-kube-api-access-clf9v\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.209154 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.208993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-dshm\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.209154 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.209010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tmp-dir\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.209154 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.209037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tls-certs\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.310560 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.310525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tls-certs\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.310741 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.310720    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-model-cache\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.310827 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.310763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-home\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.310891 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.310843    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/0c3ccdb5-fd8f-401b-b831-f17bda5db162-kube-api-access-clf9v\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.310891 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.310872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-dshm\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.311008 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.310903    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tmp-dir\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.311177 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.311155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-model-cache\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.311292 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.311196    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-home\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.311292 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.311242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tmp-dir\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.313002 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.312977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tls-certs\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.313097 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.313052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-dshm\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.320461 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.320409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/0c3ccdb5-fd8f-401b-b831-f17bda5db162-kube-api-access-clf9v\") pod \"auth-test-model-d96a19-kserve-67875d9d55-jgh6g\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") " pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.379033 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.379007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:13:59.517663 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.517633    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"]
Jul 28 15:13:59.519150 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:13:59.519124    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c3ccdb5_fd8f_401b_b831_f17bda5db162.slice/crio-a402c9952a9ecb92783ddd6ae463314fa5d85f6c184e3f8bda6af42cb2debf80 WatchSource:0}: Error finding container a402c9952a9ecb92783ddd6ae463314fa5d85f6c184e3f8bda6af42cb2debf80: Status 404 returned error can't find the container with id a402c9952a9ecb92783ddd6ae463314fa5d85f6c184e3f8bda6af42cb2debf80
Jul 28 15:13:59.996494 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.996460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g" event={"ID":"0c3ccdb5-fd8f-401b-b831-f17bda5db162","Type":"ContainerStarted","Data":"01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13"}
Jul 28 15:13:59.996494 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:13:59.996498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g" event={"ID":"0c3ccdb5-fd8f-401b-b831-f17bda5db162","Type":"ContainerStarted","Data":"a402c9952a9ecb92783ddd6ae463314fa5d85f6c184e3f8bda6af42cb2debf80"}
Jul 28 15:14:00.017786 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:00.017737    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g" podStartSLOduration=1.017720529 podStartE2EDuration="1.017720529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:14:00.015722296 +0000 UTC m=+3310.303583238" watchObservedRunningTime="2026-07-28 15:14:00.017720529 +0000 UTC m=+3310.305581473"
Jul 28 15:14:09.380138 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:09.380096    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:14:09.380549 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:09.380159    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:14:09.392376 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:09.392352    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:14:10.055283 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:10.055255    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:14:21.907251 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:21.907138    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"]
Jul 28 15:14:21.911069 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:21.911046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:21.916373 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:21.916350    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"openshift-service-ca.crt\""
Jul 28 15:14:21.917380 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:21.917361    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"auth-test-model-3edd55-kserve-self-signed-certs\""
Jul 28 15:14:21.920044 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:21.920028    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"kube-root-ca.crt\""
Jul 28 15:14:21.920191 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:21.920155    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"default-dockercfg-kszxh\""
Jul 28 15:14:21.931760 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:21.931738    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"]
Jul 28 15:14:22.020275 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.020203    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad10c959-eabb-4676-ae36-df627a7144f3-tls-certs\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.020449 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.020328    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-model-cache\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.020449 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.020359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-dshm\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.020449 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.020378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-tmp-dir\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.020576 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.020491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-home\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.020576 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.020550    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4llzb\" (UniqueName: \"kubernetes.io/projected/ad10c959-eabb-4676-ae36-df627a7144f3-kube-api-access-4llzb\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.121717 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.121687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-home\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.121911 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.121735    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4llzb\" (UniqueName: \"kubernetes.io/projected/ad10c959-eabb-4676-ae36-df627a7144f3-kube-api-access-4llzb\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.121911 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.121849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad10c959-eabb-4676-ae36-df627a7144f3-tls-certs\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.121911 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.121900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-model-cache\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.122120 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.121923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-dshm\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.122120 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.121943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-tmp-dir\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.122209 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.122125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-home\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.122287 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.122254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-model-cache\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.122287 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.122266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-tmp-dir\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.124105 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.124085    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-dshm\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.124349 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.124333    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad10c959-eabb-4676-ae36-df627a7144f3-tls-certs\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.134579 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.134550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4llzb\" (UniqueName: \"kubernetes.io/projected/ad10c959-eabb-4676-ae36-df627a7144f3-kube-api-access-4llzb\") pod \"auth-test-model-3edd55-kserve-67c768f5bc-djr5b\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") " pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.220374 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.220349    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:22.358128 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:22.358101    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"]
Jul 28 15:14:22.359969 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:14:22.359944    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad10c959_eabb_4676_ae36_df627a7144f3.slice/crio-3d7287d0086be41f1dfedc7f038dd361865fbba3c79ceebba0be9e3f6c931985 WatchSource:0}: Error finding container 3d7287d0086be41f1dfedc7f038dd361865fbba3c79ceebba0be9e3f6c931985: Status 404 returned error can't find the container with id 3d7287d0086be41f1dfedc7f038dd361865fbba3c79ceebba0be9e3f6c931985
Jul 28 15:14:23.105948 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:23.105915    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b" event={"ID":"ad10c959-eabb-4676-ae36-df627a7144f3","Type":"ContainerStarted","Data":"9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e"}
Jul 28 15:14:23.105948 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:23.105955    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b" event={"ID":"ad10c959-eabb-4676-ae36-df627a7144f3","Type":"ContainerStarted","Data":"3d7287d0086be41f1dfedc7f038dd361865fbba3c79ceebba0be9e3f6c931985"}
Jul 28 15:14:23.149510 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:23.149468    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b" podStartSLOduration=2.149453981 podStartE2EDuration="2.149453981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:14:23.149343104 +0000 UTC m=+3333.437204047" watchObservedRunningTime="2026-07-28 15:14:23.149453981 +0000 UTC m=+3333.437314924"
Jul 28 15:14:32.220857 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:32.220823    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:32.221318 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:32.220878    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:32.236250 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:32.236205    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:14:33.166875 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:14:33.166841    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:18:29.617383 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:29.617343    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"]
Jul 28 15:18:30.506151 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:30.506116    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"]
Jul 28 15:18:44.636278 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:44.636202    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g" podUID="0c3ccdb5-fd8f-401b-b831-f17bda5db162" containerName="main" containerID="cri-o://01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13" gracePeriod=45
Jul 28 15:18:44.888834 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:44.888776    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:18:45.004447 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004414    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tmp-dir\") pod \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") "
Jul 28 15:18:45.004627 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004466    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/0c3ccdb5-fd8f-401b-b831-f17bda5db162-kube-api-access-clf9v\") pod \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") "
Jul 28 15:18:45.004627 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004490    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-home\") pod \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") "
Jul 28 15:18:45.004627 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004523    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-model-cache\") pod \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") "
Jul 28 15:18:45.004627 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004582    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tls-certs\") pod \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") "
Jul 28 15:18:45.004846 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004629    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-dshm\") pod \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\" (UID: \"0c3ccdb5-fd8f-401b-b831-f17bda5db162\") "
Jul 28 15:18:45.004846 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004703    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c3ccdb5-fd8f-401b-b831-f17bda5db162" (UID: "0c3ccdb5-fd8f-401b-b831-f17bda5db162"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.004846 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004819    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-home" (OuterVolumeSpecName: "home") pod "0c3ccdb5-fd8f-401b-b831-f17bda5db162" (UID: "0c3ccdb5-fd8f-401b-b831-f17bda5db162"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.004846 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004830    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c3ccdb5-fd8f-401b-b831-f17bda5db162" (UID: "0c3ccdb5-fd8f-401b-b831-f17bda5db162"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.005029 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004952    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.005029 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004971    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.005029 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.004983    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.006842 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.006804    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c3ccdb5-fd8f-401b-b831-f17bda5db162" (UID: "0c3ccdb5-fd8f-401b-b831-f17bda5db162"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:18:45.006956 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.006897    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-dshm" (OuterVolumeSpecName: "dshm") pod "0c3ccdb5-fd8f-401b-b831-f17bda5db162" (UID: "0c3ccdb5-fd8f-401b-b831-f17bda5db162"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.006956 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.006917    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c3ccdb5-fd8f-401b-b831-f17bda5db162-kube-api-access-clf9v" (OuterVolumeSpecName: "kube-api-access-clf9v") pod "0c3ccdb5-fd8f-401b-b831-f17bda5db162" (UID: "0c3ccdb5-fd8f-401b-b831-f17bda5db162"). InnerVolumeSpecName "kube-api-access-clf9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:18:45.106221 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.106191    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c3ccdb5-fd8f-401b-b831-f17bda5db162-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.106386 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.106238    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/0c3ccdb5-fd8f-401b-b831-f17bda5db162-kube-api-access-clf9v\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.106386 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.106248    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c3ccdb5-fd8f-401b-b831-f17bda5db162-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.396503 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.396463    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c3ccdb5-fd8f-401b-b831-f17bda5db162" containerID="01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13" exitCode=0
Jul 28 15:18:45.396690 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.396513    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"
Jul 28 15:18:45.396690 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.396554    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g" event={"ID":"0c3ccdb5-fd8f-401b-b831-f17bda5db162","Type":"ContainerDied","Data":"01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13"}
Jul 28 15:18:45.396690 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.396595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g" event={"ID":"0c3ccdb5-fd8f-401b-b831-f17bda5db162","Type":"ContainerDied","Data":"a402c9952a9ecb92783ddd6ae463314fa5d85f6c184e3f8bda6af42cb2debf80"}
Jul 28 15:18:45.396690 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.396613    2579 scope.go:117] "RemoveContainer" containerID="01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13"
Jul 28 15:18:45.405627 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.405610    2579 scope.go:117] "RemoveContainer" containerID="01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13"
Jul 28 15:18:45.405884 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:18:45.405863    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13\": container with ID starting with 01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13 not found: ID does not exist" containerID="01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13"
Jul 28 15:18:45.405939 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.405894    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13"} err="failed to get container status \"01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13\": rpc error: code = NotFound desc = could not find container \"01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13\": container with ID starting with 01bef8d4025819c42fe11e1459504e05442b0daa6c6212d2f80ef1230e33eb13 not found: ID does not exist"
Jul 28 15:18:45.440162 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.440134    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"]
Jul 28 15:18:45.446619 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.446598    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/auth-test-model-d96a19-kserve-67875d9d55-jgh6g"]
Jul 28 15:18:45.524947 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.524918    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b" podUID="ad10c959-eabb-4676-ae36-df627a7144f3" containerName="main" containerID="cri-o://9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e" gracePeriod=45
Jul 28 15:18:45.776055 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.776033    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:18:45.813838 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.813815    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4llzb\" (UniqueName: \"kubernetes.io/projected/ad10c959-eabb-4676-ae36-df627a7144f3-kube-api-access-4llzb\") pod \"ad10c959-eabb-4676-ae36-df627a7144f3\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") "
Jul 28 15:18:45.813987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.813861    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-dshm\") pod \"ad10c959-eabb-4676-ae36-df627a7144f3\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") "
Jul 28 15:18:45.813987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.813883    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-home\") pod \"ad10c959-eabb-4676-ae36-df627a7144f3\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") "
Jul 28 15:18:45.813987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.813916    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-model-cache\") pod \"ad10c959-eabb-4676-ae36-df627a7144f3\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") "
Jul 28 15:18:45.813987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.813963    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad10c959-eabb-4676-ae36-df627a7144f3-tls-certs\") pod \"ad10c959-eabb-4676-ae36-df627a7144f3\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") "
Jul 28 15:18:45.814251 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.814036    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-tmp-dir\") pod \"ad10c959-eabb-4676-ae36-df627a7144f3\" (UID: \"ad10c959-eabb-4676-ae36-df627a7144f3\") "
Jul 28 15:18:45.814251 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.814198    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad10c959-eabb-4676-ae36-df627a7144f3" (UID: "ad10c959-eabb-4676-ae36-df627a7144f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.814251 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.814232    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-home" (OuterVolumeSpecName: "home") pod "ad10c959-eabb-4676-ae36-df627a7144f3" (UID: "ad10c959-eabb-4676-ae36-df627a7144f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.814422 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.814355    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad10c959-eabb-4676-ae36-df627a7144f3" (UID: "ad10c959-eabb-4676-ae36-df627a7144f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.814422 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.814409    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.814538 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.814429    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.816263 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.816242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad10c959-eabb-4676-ae36-df627a7144f3-kube-api-access-4llzb" (OuterVolumeSpecName: "kube-api-access-4llzb") pod "ad10c959-eabb-4676-ae36-df627a7144f3" (UID: "ad10c959-eabb-4676-ae36-df627a7144f3"). InnerVolumeSpecName "kube-api-access-4llzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:18:45.816370 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.816328    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad10c959-eabb-4676-ae36-df627a7144f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad10c959-eabb-4676-ae36-df627a7144f3" (UID: "ad10c959-eabb-4676-ae36-df627a7144f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:18:45.816370 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.816358    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-dshm" (OuterVolumeSpecName: "dshm") pod "ad10c959-eabb-4676-ae36-df627a7144f3" (UID: "ad10c959-eabb-4676-ae36-df627a7144f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:18:45.915066 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.914996    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.915066 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.915023    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad10c959-eabb-4676-ae36-df627a7144f3-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.915066 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.915033    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad10c959-eabb-4676-ae36-df627a7144f3-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:45.915066 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:45.915041    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4llzb\" (UniqueName: \"kubernetes.io/projected/ad10c959-eabb-4676-ae36-df627a7144f3-kube-api-access-4llzb\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:18:46.192992 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.192950    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c3ccdb5-fd8f-401b-b831-f17bda5db162" path="/var/lib/kubelet/pods/0c3ccdb5-fd8f-401b-b831-f17bda5db162/volumes"
Jul 28 15:18:46.403573 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.403538    2579 generic.go:356] "Generic (PLEG): container finished" podID="ad10c959-eabb-4676-ae36-df627a7144f3" containerID="9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e" exitCode=0
Jul 28 15:18:46.403739 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.403584    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"
Jul 28 15:18:46.403739 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.403626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b" event={"ID":"ad10c959-eabb-4676-ae36-df627a7144f3","Type":"ContainerDied","Data":"9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e"}
Jul 28 15:18:46.403739 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.403666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b" event={"ID":"ad10c959-eabb-4676-ae36-df627a7144f3","Type":"ContainerDied","Data":"3d7287d0086be41f1dfedc7f038dd361865fbba3c79ceebba0be9e3f6c931985"}
Jul 28 15:18:46.403739 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.403684    2579 scope.go:117] "RemoveContainer" containerID="9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e"
Jul 28 15:18:46.411877 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.411853    2579 scope.go:117] "RemoveContainer" containerID="9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e"
Jul 28 15:18:46.412125 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:18:46.412106    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e\": container with ID starting with 9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e not found: ID does not exist" containerID="9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e"
Jul 28 15:18:46.412188 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.412136    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e"} err="failed to get container status \"9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e\": rpc error: code = NotFound desc = could not find container \"9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e\": container with ID starting with 9d66c463bced1594ac5bfa9ad4465f8cee336b398c081a4c631fe7c8faa11e3e not found: ID does not exist"
Jul 28 15:18:46.430141 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.430118    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"]
Jul 28 15:18:46.437455 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:46.437433    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/auth-test-model-3edd55-kserve-67c768f5bc-djr5b"]
Jul 28 15:18:48.192163 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:48.192130    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad10c959-eabb-4676-ae36-df627a7144f3" path="/var/lib/kubelet/pods/ad10c959-eabb-4676-ae36-df627a7144f3/volumes"
Jul 28 15:18:50.367127 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:50.367028    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:18:50.382242 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:18:50.379334    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:19:08.948457 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.948419    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"]
Jul 28 15:19:08.949026 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.948993    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c3ccdb5-fd8f-401b-b831-f17bda5db162" containerName="main"
Jul 28 15:19:08.949026 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.949012    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c3ccdb5-fd8f-401b-b831-f17bda5db162" containerName="main"
Jul 28 15:19:08.949026 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.949025    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad10c959-eabb-4676-ae36-df627a7144f3" containerName="main"
Jul 28 15:19:08.949198 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.949035    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad10c959-eabb-4676-ae36-df627a7144f3" containerName="main"
Jul 28 15:19:08.949198 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.949124    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c3ccdb5-fd8f-401b-b831-f17bda5db162" containerName="main"
Jul 28 15:19:08.949198 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.949136    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad10c959-eabb-4676-ae36-df627a7144f3" containerName="main"
Jul 28 15:19:08.952110 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.952090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:08.956947 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.956925    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"openshift-service-ca.crt\""
Jul 28 15:19:08.957039 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.956925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"default-dockercfg-sqhrp\""
Jul 28 15:19:08.957039 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.956980    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"sub-test-model-d96a19-kserve-self-signed-certs\""
Jul 28 15:19:08.957601 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.957581    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"kube-root-ca.crt\""
Jul 28 15:19:08.969827 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:08.969807    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"]
Jul 28 15:19:09.033462 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.033427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd459216-f515-4ab6-8e6a-f3e098522cc9-tls-certs\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.033603 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.033472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-dshm\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.033603 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.033575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfcvm\" (UniqueName: \"kubernetes.io/projected/bd459216-f515-4ab6-8e6a-f3e098522cc9-kube-api-access-lfcvm\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.033703 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.033603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-home\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.033703 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.033628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-tmp-dir\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.033703 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.033652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-model-cache\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.134545 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.134513    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd459216-f515-4ab6-8e6a-f3e098522cc9-tls-certs\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.134545 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.134546    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-dshm\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.134777 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.134706    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfcvm\" (UniqueName: \"kubernetes.io/projected/bd459216-f515-4ab6-8e6a-f3e098522cc9-kube-api-access-lfcvm\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.134777 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.134752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-home\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.134907 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.134799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-tmp-dir\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.134907 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.134834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-model-cache\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.135175 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.135146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-home\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.135295 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.135200    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-tmp-dir\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.135295 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.135262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-model-cache\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.136747 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.136720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-dshm\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.137141 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.137119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd459216-f515-4ab6-8e6a-f3e098522cc9-tls-certs\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.144177 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.144147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfcvm\" (UniqueName: \"kubernetes.io/projected/bd459216-f515-4ab6-8e6a-f3e098522cc9-kube-api-access-lfcvm\") pod \"sub-test-model-d96a19-kserve-7cd95d4979-rt2qg\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") " pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.261680 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.261603    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:09.403119 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.403088    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"]
Jul 28 15:19:09.406637 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:19:09.406607    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd459216_f515_4ab6_8e6a_f3e098522cc9.slice/crio-816d07a9b4327e6098c361af4b8467a45ebd71ffa946a62996dceb7aada8f3bf WatchSource:0}: Error finding container 816d07a9b4327e6098c361af4b8467a45ebd71ffa946a62996dceb7aada8f3bf: Status 404 returned error can't find the container with id 816d07a9b4327e6098c361af4b8467a45ebd71ffa946a62996dceb7aada8f3bf
Jul 28 15:19:09.522818 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.522714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg" event={"ID":"bd459216-f515-4ab6-8e6a-f3e098522cc9","Type":"ContainerStarted","Data":"58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079"}
Jul 28 15:19:09.522818 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.522760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg" event={"ID":"bd459216-f515-4ab6-8e6a-f3e098522cc9","Type":"ContainerStarted","Data":"816d07a9b4327e6098c361af4b8467a45ebd71ffa946a62996dceb7aada8f3bf"}
Jul 28 15:19:09.548018 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:09.547968    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg" podStartSLOduration=1.54795313 podStartE2EDuration="1.54795313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:19:09.545226844 +0000 UTC m=+3619.833087777" watchObservedRunningTime="2026-07-28 15:19:09.54795313 +0000 UTC m=+3619.835814072"
Jul 28 15:19:19.262309 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:19.262274    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:19.262814 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:19.262327    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:19.276657 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:19.276635    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:19.584841 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:19.584752    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:19:40.238815 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.238774    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"]
Jul 28 15:19:40.242182 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.242165    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.252160 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.252142    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"default-dockercfg-kszxh\""
Jul 28 15:19:40.252281 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.252174    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"openshift-service-ca.crt\""
Jul 28 15:19:40.252281 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.252260    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"kube-root-ca.crt\""
Jul 28 15:19:40.252405 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.252260    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"sub-test-model-3edd55-kserve-self-signed-certs\""
Jul 28 15:19:40.271458 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.271427    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"]
Jul 28 15:19:40.312482 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.312454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-model-cache\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.312615 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.312510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50db1987-91b5-47e5-a0ee-944a94a7f776-tls-certs\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.312615 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.312567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5d44\" (UniqueName: \"kubernetes.io/projected/50db1987-91b5-47e5-a0ee-944a94a7f776-kube-api-access-f5d44\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.312615 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.312593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-dshm\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.312748 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.312640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-home\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.312748 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.312694    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-tmp-dir\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414081 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414057    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-model-cache\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414234 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50db1987-91b5-47e5-a0ee-944a94a7f776-tls-certs\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414234 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414122    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5d44\" (UniqueName: \"kubernetes.io/projected/50db1987-91b5-47e5-a0ee-944a94a7f776-kube-api-access-f5d44\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414362 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-dshm\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414362 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-home\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414475 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-tmp-dir\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414475 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-model-cache\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414632 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-home\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.414674 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.414651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-tmp-dir\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.416384 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.416361    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-dshm\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.416586 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.416568    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50db1987-91b5-47e5-a0ee-944a94a7f776-tls-certs\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.433577 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.433554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5d44\" (UniqueName: \"kubernetes.io/projected/50db1987-91b5-47e5-a0ee-944a94a7f776-kube-api-access-f5d44\") pod \"sub-test-model-3edd55-kserve-7598944587-cn4xq\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") " pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.551899 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.551819    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:40.688312 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:40.688284    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"]
Jul 28 15:19:40.690931 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:19:40.690904    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50db1987_91b5_47e5_a0ee_944a94a7f776.slice/crio-d12289499b868ac2e2099e9eed61be1dc0eae1e08db36bf3d5cee7e46991d9d6 WatchSource:0}: Error finding container d12289499b868ac2e2099e9eed61be1dc0eae1e08db36bf3d5cee7e46991d9d6: Status 404 returned error can't find the container with id d12289499b868ac2e2099e9eed61be1dc0eae1e08db36bf3d5cee7e46991d9d6
Jul 28 15:19:41.674832 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:41.674793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq" event={"ID":"50db1987-91b5-47e5-a0ee-944a94a7f776","Type":"ContainerStarted","Data":"e0c257174f276d54c3105d999ad2473a9dd17197652bc2dbe401ede45770db11"}
Jul 28 15:19:41.675301 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:41.674839    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq" event={"ID":"50db1987-91b5-47e5-a0ee-944a94a7f776","Type":"ContainerStarted","Data":"d12289499b868ac2e2099e9eed61be1dc0eae1e08db36bf3d5cee7e46991d9d6"}
Jul 28 15:19:41.694527 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:41.694485    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq" podStartSLOduration=1.6944705359999999 podStartE2EDuration="1.694470536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:19:41.692199932 +0000 UTC m=+3651.980060876" watchObservedRunningTime="2026-07-28 15:19:41.694470536 +0000 UTC m=+3651.982331479"
Jul 28 15:19:50.552129 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:50.552096    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:50.552655 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:50.552143    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:50.564684 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:50.564662    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:19:50.731750 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:19:50.731723    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:21:25.132476 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:25.132442    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"]
Jul 28 15:21:25.978875 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:25.978842    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"]
Jul 28 15:21:34.558742 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.558702    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"]
Jul 28 15:21:34.562978 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.562954    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.565677 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.565656    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d96a19\"/\"rate-test-model-d96a19-kserve-self-signed-certs\""
Jul 28 15:21:34.572575 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.572553    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"]
Jul 28 15:21:34.657478 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.657449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-home\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.657635 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.657509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-tmp-dir\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.657635 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.657559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84xjh\" (UniqueName: \"kubernetes.io/projected/fce7150b-d447-48c4-af9d-194c1821b192-kube-api-access-84xjh\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.657635 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.657597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fce7150b-d447-48c4-af9d-194c1821b192-tls-certs\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.657635 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.657627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-dshm\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.657820 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.657702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-model-cache\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759060 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759031    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-home\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759230 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-tmp-dir\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759230 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84xjh\" (UniqueName: \"kubernetes.io/projected/fce7150b-d447-48c4-af9d-194c1821b192-kube-api-access-84xjh\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759230 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fce7150b-d447-48c4-af9d-194c1821b192-tls-certs\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759230 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-dshm\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759477 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-model-cache\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759585 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759520    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-home\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759585 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-tmp-dir\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.759769 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.759678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-model-cache\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.761537 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.761514    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-dshm\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.761987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.761967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fce7150b-d447-48c4-af9d-194c1821b192-tls-certs\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.767736 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.767716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84xjh\" (UniqueName: \"kubernetes.io/projected/fce7150b-d447-48c4-af9d-194c1821b192-kube-api-access-84xjh\") pod \"rate-test-model-d96a19-kserve-75dd975cff-25txk\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") " pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:34.873071 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:34.873009    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:35.007687 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:35.007662    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"]
Jul 28 15:21:35.009619 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:21:35.009587    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfce7150b_d447_48c4_af9d_194c1821b192.slice/crio-6e5ffe13bff5cdb100f6cd645c80ea1494e73e495a0103ab507f8df5c1c528c5 WatchSource:0}: Error finding container 6e5ffe13bff5cdb100f6cd645c80ea1494e73e495a0103ab507f8df5c1c528c5: Status 404 returned error can't find the container with id 6e5ffe13bff5cdb100f6cd645c80ea1494e73e495a0103ab507f8df5c1c528c5
Jul 28 15:21:35.232750 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:35.232710    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk" event={"ID":"fce7150b-d447-48c4-af9d-194c1821b192","Type":"ContainerStarted","Data":"5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81"}
Jul 28 15:21:35.232950 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:35.232765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk" event={"ID":"fce7150b-d447-48c4-af9d-194c1821b192","Type":"ContainerStarted","Data":"6e5ffe13bff5cdb100f6cd645c80ea1494e73e495a0103ab507f8df5c1c528c5"}
Jul 28 15:21:35.252595 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:35.252542    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk" podStartSLOduration=1.252523343 podStartE2EDuration="1.252523343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:21:35.252257064 +0000 UTC m=+3765.540118007" watchObservedRunningTime="2026-07-28 15:21:35.252523343 +0000 UTC m=+3765.540384287"
Jul 28 15:21:40.151199 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.151142    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg" podUID="bd459216-f515-4ab6-8e6a-f3e098522cc9" containerName="main" containerID="cri-o://58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079" gracePeriod=45
Jul 28 15:21:40.406902 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.406844    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:21:40.520157 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520122    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-home\") pod \"bd459216-f515-4ab6-8e6a-f3e098522cc9\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") "
Jul 28 15:21:40.520157 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520154    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-tmp-dir\") pod \"bd459216-f515-4ab6-8e6a-f3e098522cc9\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") "
Jul 28 15:21:40.520374 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520171    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd459216-f515-4ab6-8e6a-f3e098522cc9-tls-certs\") pod \"bd459216-f515-4ab6-8e6a-f3e098522cc9\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") "
Jul 28 15:21:40.520374 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520240    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfcvm\" (UniqueName: \"kubernetes.io/projected/bd459216-f515-4ab6-8e6a-f3e098522cc9-kube-api-access-lfcvm\") pod \"bd459216-f515-4ab6-8e6a-f3e098522cc9\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") "
Jul 28 15:21:40.520374 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520289    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-model-cache\") pod \"bd459216-f515-4ab6-8e6a-f3e098522cc9\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") "
Jul 28 15:21:40.520374 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520307    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-dshm\") pod \"bd459216-f515-4ab6-8e6a-f3e098522cc9\" (UID: \"bd459216-f515-4ab6-8e6a-f3e098522cc9\") "
Jul 28 15:21:40.520560 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520444    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-home" (OuterVolumeSpecName: "home") pod "bd459216-f515-4ab6-8e6a-f3e098522cc9" (UID: "bd459216-f515-4ab6-8e6a-f3e098522cc9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:40.520560 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520455    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd459216-f515-4ab6-8e6a-f3e098522cc9" (UID: "bd459216-f515-4ab6-8e6a-f3e098522cc9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:40.520669 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520591    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd459216-f515-4ab6-8e6a-f3e098522cc9" (UID: "bd459216-f515-4ab6-8e6a-f3e098522cc9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:40.520669 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520604    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:40.520669 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.520644    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:40.522353 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.522333    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-dshm" (OuterVolumeSpecName: "dshm") pod "bd459216-f515-4ab6-8e6a-f3e098522cc9" (UID: "bd459216-f515-4ab6-8e6a-f3e098522cc9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:40.522441 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.522417    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd459216-f515-4ab6-8e6a-f3e098522cc9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd459216-f515-4ab6-8e6a-f3e098522cc9" (UID: "bd459216-f515-4ab6-8e6a-f3e098522cc9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:21:40.522498 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.522480    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd459216-f515-4ab6-8e6a-f3e098522cc9-kube-api-access-lfcvm" (OuterVolumeSpecName: "kube-api-access-lfcvm") pod "bd459216-f515-4ab6-8e6a-f3e098522cc9" (UID: "bd459216-f515-4ab6-8e6a-f3e098522cc9"). InnerVolumeSpecName "kube-api-access-lfcvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:21:40.621410 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.621377    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd459216-f515-4ab6-8e6a-f3e098522cc9-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:40.621410 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.621406    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfcvm\" (UniqueName: \"kubernetes.io/projected/bd459216-f515-4ab6-8e6a-f3e098522cc9-kube-api-access-lfcvm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:40.621587 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.621419    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:40.621587 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.621431    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd459216-f515-4ab6-8e6a-f3e098522cc9-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:40.997536 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:40.997473    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq" podUID="50db1987-91b5-47e5-a0ee-944a94a7f776" containerName="main" containerID="cri-o://e0c257174f276d54c3105d999ad2473a9dd17197652bc2dbe401ede45770db11" gracePeriod=45
Jul 28 15:21:41.263567 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.263543    2579 generic.go:356] "Generic (PLEG): container finished" podID="50db1987-91b5-47e5-a0ee-944a94a7f776" containerID="e0c257174f276d54c3105d999ad2473a9dd17197652bc2dbe401ede45770db11" exitCode=0
Jul 28 15:21:41.263884 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.263613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq" event={"ID":"50db1987-91b5-47e5-a0ee-944a94a7f776","Type":"ContainerDied","Data":"e0c257174f276d54c3105d999ad2473a9dd17197652bc2dbe401ede45770db11"}
Jul 28 15:21:41.263884 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.263650    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq" event={"ID":"50db1987-91b5-47e5-a0ee-944a94a7f776","Type":"ContainerDied","Data":"d12289499b868ac2e2099e9eed61be1dc0eae1e08db36bf3d5cee7e46991d9d6"}
Jul 28 15:21:41.263884 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.263668    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d12289499b868ac2e2099e9eed61be1dc0eae1e08db36bf3d5cee7e46991d9d6"
Jul 28 15:21:41.264955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.264934    2579 generic.go:356] "Generic (PLEG): container finished" podID="bd459216-f515-4ab6-8e6a-f3e098522cc9" containerID="58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079" exitCode=0
Jul 28 15:21:41.265052 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.264962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg" event={"ID":"bd459216-f515-4ab6-8e6a-f3e098522cc9","Type":"ContainerDied","Data":"58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079"}
Jul 28 15:21:41.265052 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.264976    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"
Jul 28 15:21:41.265052 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.264984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg" event={"ID":"bd459216-f515-4ab6-8e6a-f3e098522cc9","Type":"ContainerDied","Data":"816d07a9b4327e6098c361af4b8467a45ebd71ffa946a62996dceb7aada8f3bf"}
Jul 28 15:21:41.265052 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.265004    2579 scope.go:117] "RemoveContainer" containerID="58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079"
Jul 28 15:21:41.268561 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.268544    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:21:41.273553 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.273539    2579 scope.go:117] "RemoveContainer" containerID="58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079"
Jul 28 15:21:41.273803 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:21:41.273784    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079\": container with ID starting with 58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079 not found: ID does not exist" containerID="58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079"
Jul 28 15:21:41.273864 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.273814    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079"} err="failed to get container status \"58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079\": rpc error: code = NotFound desc = could not find container \"58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079\": container with ID starting with 58edffb96c7f8310dba141423fd849f470757b5d5a787f102ef5bea32d729079 not found: ID does not exist"
Jul 28 15:21:41.327511 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327483    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-model-cache\") pod \"50db1987-91b5-47e5-a0ee-944a94a7f776\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") "
Jul 28 15:21:41.327644 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327534    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-home\") pod \"50db1987-91b5-47e5-a0ee-944a94a7f776\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") "
Jul 28 15:21:41.327644 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50db1987-91b5-47e5-a0ee-944a94a7f776-tls-certs\") pod \"50db1987-91b5-47e5-a0ee-944a94a7f776\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") "
Jul 28 15:21:41.327644 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327631    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-dshm\") pod \"50db1987-91b5-47e5-a0ee-944a94a7f776\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") "
Jul 28 15:21:41.327817 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327690    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5d44\" (UniqueName: \"kubernetes.io/projected/50db1987-91b5-47e5-a0ee-944a94a7f776-kube-api-access-f5d44\") pod \"50db1987-91b5-47e5-a0ee-944a94a7f776\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") "
Jul 28 15:21:41.327817 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327757    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-model-cache" (OuterVolumeSpecName: "model-cache") pod "50db1987-91b5-47e5-a0ee-944a94a7f776" (UID: "50db1987-91b5-47e5-a0ee-944a94a7f776"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:41.327817 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327765    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-tmp-dir\") pod \"50db1987-91b5-47e5-a0ee-944a94a7f776\" (UID: \"50db1987-91b5-47e5-a0ee-944a94a7f776\") "
Jul 28 15:21:41.327976 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.327953    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-home" (OuterVolumeSpecName: "home") pod "50db1987-91b5-47e5-a0ee-944a94a7f776" (UID: "50db1987-91b5-47e5-a0ee-944a94a7f776"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:41.328140 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.328120    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:41.328238 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.328144    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:41.328316 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.328292    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50db1987-91b5-47e5-a0ee-944a94a7f776" (UID: "50db1987-91b5-47e5-a0ee-944a94a7f776"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:41.330113 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.330088    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50db1987-91b5-47e5-a0ee-944a94a7f776-kube-api-access-f5d44" (OuterVolumeSpecName: "kube-api-access-f5d44") pod "50db1987-91b5-47e5-a0ee-944a94a7f776" (UID: "50db1987-91b5-47e5-a0ee-944a94a7f776"). InnerVolumeSpecName "kube-api-access-f5d44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:21:41.330321 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.330299    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50db1987-91b5-47e5-a0ee-944a94a7f776-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50db1987-91b5-47e5-a0ee-944a94a7f776" (UID: "50db1987-91b5-47e5-a0ee-944a94a7f776"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:21:41.330431 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.330317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-dshm" (OuterVolumeSpecName: "dshm") pod "50db1987-91b5-47e5-a0ee-944a94a7f776" (UID: "50db1987-91b5-47e5-a0ee-944a94a7f776"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:21:41.353103 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.353081    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"]
Jul 28 15:21:41.363112 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.363089    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/sub-test-model-d96a19-kserve-7cd95d4979-rt2qg"]
Jul 28 15:21:41.429500 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.429473    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:41.429500 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.429495    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50db1987-91b5-47e5-a0ee-944a94a7f776-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:41.429636 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.429507    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50db1987-91b5-47e5-a0ee-944a94a7f776-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:41.429636 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:41.429518    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5d44\" (UniqueName: \"kubernetes.io/projected/50db1987-91b5-47e5-a0ee-944a94a7f776-kube-api-access-f5d44\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:21:42.193136 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:42.193052    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd459216-f515-4ab6-8e6a-f3e098522cc9" path="/var/lib/kubelet/pods/bd459216-f515-4ab6-8e6a-f3e098522cc9/volumes"
Jul 28 15:21:42.271754 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:42.271719    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"
Jul 28 15:21:42.291643 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:42.291613    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"]
Jul 28 15:21:42.296483 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:42.296459    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/sub-test-model-3edd55-kserve-7598944587-cn4xq"]
Jul 28 15:21:44.193411 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:44.193378    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50db1987-91b5-47e5-a0ee-944a94a7f776" path="/var/lib/kubelet/pods/50db1987-91b5-47e5-a0ee-944a94a7f776/volumes"
Jul 28 15:21:44.874029 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:44.873996    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:44.874241 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:44.874053    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:44.886186 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:44.886165    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:45.296977 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:45.296949    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:21:56.491938 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.491909    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"]
Jul 28 15:21:56.492432 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.492292    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50db1987-91b5-47e5-a0ee-944a94a7f776" containerName="main"
Jul 28 15:21:56.492432 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.492303    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="50db1987-91b5-47e5-a0ee-944a94a7f776" containerName="main"
Jul 28 15:21:56.492432 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.492330    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd459216-f515-4ab6-8e6a-f3e098522cc9" containerName="main"
Jul 28 15:21:56.492432 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.492335    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd459216-f515-4ab6-8e6a-f3e098522cc9" containerName="main"
Jul 28 15:21:56.492432 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.492404    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="50db1987-91b5-47e5-a0ee-944a94a7f776" containerName="main"
Jul 28 15:21:56.492432 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.492412    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd459216-f515-4ab6-8e6a-f3e098522cc9" containerName="main"
Jul 28 15:21:56.496703 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.496688    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.503488 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.503463    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"rate-test-model-3edd55-kserve-self-signed-certs\""
Jul 28 15:21:56.503623 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.503535    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"kube-root-ca.crt\""
Jul 28 15:21:56.503699 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.503684    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"openshift-service-ca.crt\""
Jul 28 15:21:56.503799 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.503782    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3edd55\"/\"default-dockercfg-kszxh\""
Jul 28 15:21:56.512636 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.512615    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"]
Jul 28 15:21:56.672626 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.672592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s549c\" (UniqueName: \"kubernetes.io/projected/be12b2fe-504f-4247-b371-cfc42318858e-kube-api-access-s549c\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.672819 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.672645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-model-cache\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.672819 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.672671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be12b2fe-504f-4247-b371-cfc42318858e-tls-certs\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.672819 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.672750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-tmp-dir\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.672819 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.672782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-dshm\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.672976 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.672820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-home\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773392 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-model-cache\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773392 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be12b2fe-504f-4247-b371-cfc42318858e-tls-certs\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773616 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-tmp-dir\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773616 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-dshm\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773616 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-home\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773616 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s549c\" (UniqueName: \"kubernetes.io/projected/be12b2fe-504f-4247-b371-cfc42318858e-kube-api-access-s549c\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773832 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773776    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-model-cache\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773832 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-tmp-dir\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.773914 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.773856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-home\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.775640 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.775618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-dshm\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.775929 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.775910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be12b2fe-504f-4247-b371-cfc42318858e-tls-certs\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.783886 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.783864    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s549c\" (UniqueName: \"kubernetes.io/projected/be12b2fe-504f-4247-b371-cfc42318858e-kube-api-access-s549c\") pod \"rate-test-model-3edd55-kserve-75f6978987-sb2c7\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") " pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.805973 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.805951    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:21:56.942335 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:56.942310    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"]
Jul 28 15:21:56.945023 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:21:56.944992    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe12b2fe_504f_4247_b371_cfc42318858e.slice/crio-4b1aa1db0ed345c124eaeeadfdfb7deb2bbf376f8d57f2bf765e05e0024887b0 WatchSource:0}: Error finding container 4b1aa1db0ed345c124eaeeadfdfb7deb2bbf376f8d57f2bf765e05e0024887b0: Status 404 returned error can't find the container with id 4b1aa1db0ed345c124eaeeadfdfb7deb2bbf376f8d57f2bf765e05e0024887b0
Jul 28 15:21:57.350396 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:57.350354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7" event={"ID":"be12b2fe-504f-4247-b371-cfc42318858e","Type":"ContainerStarted","Data":"6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591"}
Jul 28 15:21:57.350396 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:57.350396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7" event={"ID":"be12b2fe-504f-4247-b371-cfc42318858e","Type":"ContainerStarted","Data":"4b1aa1db0ed345c124eaeeadfdfb7deb2bbf376f8d57f2bf765e05e0024887b0"}
Jul 28 15:21:57.374609 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:21:57.374562    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7" podStartSLOduration=1.3745464840000001 podStartE2EDuration="1.374546484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:21:57.373078932 +0000 UTC m=+3787.660939875" watchObservedRunningTime="2026-07-28 15:21:57.374546484 +0000 UTC m=+3787.662407476"
Jul 28 15:22:06.806327 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:22:06.806291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:22:06.806775 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:22:06.806345    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:22:06.818739 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:22:06.818715    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:22:07.414535 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:22:07.414508    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:22:56.894979 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:22:56.894939    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"]
Jul 28 15:22:57.932041 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:22:57.932010    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"]
Jul 28 15:23:11.913562 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:11.913524    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk" podUID="fce7150b-d447-48c4-af9d-194c1821b192" containerName="main" containerID="cri-o://5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81" gracePeriod=45
Jul 28 15:23:12.163383 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.163359    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:23:12.257102 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257062    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84xjh\" (UniqueName: \"kubernetes.io/projected/fce7150b-d447-48c4-af9d-194c1821b192-kube-api-access-84xjh\") pod \"fce7150b-d447-48c4-af9d-194c1821b192\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") "
Jul 28 15:23:12.257280 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-home\") pod \"fce7150b-d447-48c4-af9d-194c1821b192\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") "
Jul 28 15:23:12.257280 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-tmp-dir\") pod \"fce7150b-d447-48c4-af9d-194c1821b192\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") "
Jul 28 15:23:12.257280 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257245    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fce7150b-d447-48c4-af9d-194c1821b192-tls-certs\") pod \"fce7150b-d447-48c4-af9d-194c1821b192\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") "
Jul 28 15:23:12.257280 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257276    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-model-cache\") pod \"fce7150b-d447-48c4-af9d-194c1821b192\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") "
Jul 28 15:23:12.257504 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257299    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-dshm\") pod \"fce7150b-d447-48c4-af9d-194c1821b192\" (UID: \"fce7150b-d447-48c4-af9d-194c1821b192\") "
Jul 28 15:23:12.257504 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257434    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-home" (OuterVolumeSpecName: "home") pod "fce7150b-d447-48c4-af9d-194c1821b192" (UID: "fce7150b-d447-48c4-af9d-194c1821b192"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:12.257613 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257521    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fce7150b-d447-48c4-af9d-194c1821b192" (UID: "fce7150b-d447-48c4-af9d-194c1821b192"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:12.257613 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257573    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-model-cache" (OuterVolumeSpecName: "model-cache") pod "fce7150b-d447-48c4-af9d-194c1821b192" (UID: "fce7150b-d447-48c4-af9d-194c1821b192"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:12.257760 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257737    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:12.257890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.257764    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:12.259265 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.259242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-dshm" (OuterVolumeSpecName: "dshm") pod "fce7150b-d447-48c4-af9d-194c1821b192" (UID: "fce7150b-d447-48c4-af9d-194c1821b192"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:12.259369 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.259264    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fce7150b-d447-48c4-af9d-194c1821b192-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fce7150b-d447-48c4-af9d-194c1821b192" (UID: "fce7150b-d447-48c4-af9d-194c1821b192"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:23:12.259369 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.259342    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fce7150b-d447-48c4-af9d-194c1821b192-kube-api-access-84xjh" (OuterVolumeSpecName: "kube-api-access-84xjh") pod "fce7150b-d447-48c4-af9d-194c1821b192" (UID: "fce7150b-d447-48c4-af9d-194c1821b192"). InnerVolumeSpecName "kube-api-access-84xjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:23:12.358550 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.358522    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84xjh\" (UniqueName: \"kubernetes.io/projected/fce7150b-d447-48c4-af9d-194c1821b192-kube-api-access-84xjh\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:12.358550 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.358546    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fce7150b-d447-48c4-af9d-194c1821b192-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:12.358719 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.358557    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:12.358719 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.358565    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fce7150b-d447-48c4-af9d-194c1821b192-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:12.736602 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.736565    2579 generic.go:356] "Generic (PLEG): container finished" podID="fce7150b-d447-48c4-af9d-194c1821b192" containerID="5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81" exitCode=0
Jul 28 15:23:12.736776 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.736615    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"
Jul 28 15:23:12.736776 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.736637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk" event={"ID":"fce7150b-d447-48c4-af9d-194c1821b192","Type":"ContainerDied","Data":"5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81"}
Jul 28 15:23:12.736776 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.736675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk" event={"ID":"fce7150b-d447-48c4-af9d-194c1821b192","Type":"ContainerDied","Data":"6e5ffe13bff5cdb100f6cd645c80ea1494e73e495a0103ab507f8df5c1c528c5"}
Jul 28 15:23:12.736776 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.736696    2579 scope.go:117] "RemoveContainer" containerID="5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81"
Jul 28 15:23:12.750309 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.750280    2579 scope.go:117] "RemoveContainer" containerID="5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81"
Jul 28 15:23:12.750525 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:23:12.750506    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81\": container with ID starting with 5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81 not found: ID does not exist" containerID="5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81"
Jul 28 15:23:12.750583 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.750533    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81"} err="failed to get container status \"5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81\": rpc error: code = NotFound desc = could not find container \"5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81\": container with ID starting with 5d7377e7984604ed1491bc10c4492c1af6d7fcae41f87ea7754f6f66faacaf81 not found: ID does not exist"
Jul 28 15:23:12.780324 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.780303    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"]
Jul 28 15:23:12.783333 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.783312    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d96a19/rate-test-model-d96a19-kserve-75dd975cff-25txk"]
Jul 28 15:23:12.950642 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:12.950576    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7" podUID="be12b2fe-504f-4247-b371-cfc42318858e" containerName="main" containerID="cri-o://6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591" gracePeriod=45
Jul 28 15:23:13.199243 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.199204    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:23:13.264314 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-tmp-dir\") pod \"be12b2fe-504f-4247-b371-cfc42318858e\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") "
Jul 28 15:23:13.264488 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264336    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-model-cache\") pod \"be12b2fe-504f-4247-b371-cfc42318858e\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") "
Jul 28 15:23:13.264488 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264392    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-dshm\") pod \"be12b2fe-504f-4247-b371-cfc42318858e\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") "
Jul 28 15:23:13.264488 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264414    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-home\") pod \"be12b2fe-504f-4247-b371-cfc42318858e\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") "
Jul 28 15:23:13.264488 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264440    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s549c\" (UniqueName: \"kubernetes.io/projected/be12b2fe-504f-4247-b371-cfc42318858e-kube-api-access-s549c\") pod \"be12b2fe-504f-4247-b371-cfc42318858e\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") "
Jul 28 15:23:13.264488 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264483    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be12b2fe-504f-4247-b371-cfc42318858e-tls-certs\") pod \"be12b2fe-504f-4247-b371-cfc42318858e\" (UID: \"be12b2fe-504f-4247-b371-cfc42318858e\") "
Jul 28 15:23:13.264751 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264550    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-model-cache" (OuterVolumeSpecName: "model-cache") pod "be12b2fe-504f-4247-b371-cfc42318858e" (UID: "be12b2fe-504f-4247-b371-cfc42318858e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:13.264751 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "be12b2fe-504f-4247-b371-cfc42318858e" (UID: "be12b2fe-504f-4247-b371-cfc42318858e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:13.264751 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264698    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-home" (OuterVolumeSpecName: "home") pod "be12b2fe-504f-4247-b371-cfc42318858e" (UID: "be12b2fe-504f-4247-b371-cfc42318858e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:13.264916 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264894    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:13.264955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264922    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:13.264955 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.264936    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:13.266641 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.266617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-dshm" (OuterVolumeSpecName: "dshm") pod "be12b2fe-504f-4247-b371-cfc42318858e" (UID: "be12b2fe-504f-4247-b371-cfc42318858e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:23:13.266972 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.266956    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be12b2fe-504f-4247-b371-cfc42318858e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "be12b2fe-504f-4247-b371-cfc42318858e" (UID: "be12b2fe-504f-4247-b371-cfc42318858e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:23:13.267023 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.266978    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be12b2fe-504f-4247-b371-cfc42318858e-kube-api-access-s549c" (OuterVolumeSpecName: "kube-api-access-s549c") pod "be12b2fe-504f-4247-b371-cfc42318858e" (UID: "be12b2fe-504f-4247-b371-cfc42318858e"). InnerVolumeSpecName "kube-api-access-s549c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:23:13.366449 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.366384    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be12b2fe-504f-4247-b371-cfc42318858e-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:13.366449 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.366405    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s549c\" (UniqueName: \"kubernetes.io/projected/be12b2fe-504f-4247-b371-cfc42318858e-kube-api-access-s549c\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:13.366449 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.366415    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be12b2fe-504f-4247-b371-cfc42318858e-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:13.743311 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.743278    2579 generic.go:356] "Generic (PLEG): container finished" podID="be12b2fe-504f-4247-b371-cfc42318858e" containerID="6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591" exitCode=0
Jul 28 15:23:13.743490 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.743324    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"
Jul 28 15:23:13.743490 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.743351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7" event={"ID":"be12b2fe-504f-4247-b371-cfc42318858e","Type":"ContainerDied","Data":"6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591"}
Jul 28 15:23:13.743490 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.743388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7" event={"ID":"be12b2fe-504f-4247-b371-cfc42318858e","Type":"ContainerDied","Data":"4b1aa1db0ed345c124eaeeadfdfb7deb2bbf376f8d57f2bf765e05e0024887b0"}
Jul 28 15:23:13.743490 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.743410    2579 scope.go:117] "RemoveContainer" containerID="6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591"
Jul 28 15:23:13.752535 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.752500    2579 scope.go:117] "RemoveContainer" containerID="6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591"
Jul 28 15:23:13.752807 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:23:13.752788    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591\": container with ID starting with 6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591 not found: ID does not exist" containerID="6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591"
Jul 28 15:23:13.752872 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.752813    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591"} err="failed to get container status \"6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591\": rpc error: code = NotFound desc = could not find container \"6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591\": container with ID starting with 6beebd012d3d3aa65dac8b2b0bd6488cf34d90facf4557b13e20eeb924c92591 not found: ID does not exist"
Jul 28 15:23:13.783046 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.783020    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"]
Jul 28 15:23:13.786810 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:13.786789    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3edd55/rate-test-model-3edd55-kserve-75f6978987-sb2c7"]
Jul 28 15:23:14.192426 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.192396    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be12b2fe-504f-4247-b371-cfc42318858e" path="/var/lib/kubelet/pods/be12b2fe-504f-4247-b371-cfc42318858e/volumes"
Jul 28 15:23:14.192791 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.192781    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fce7150b-d447-48c4-af9d-194c1821b192" path="/var/lib/kubelet/pods/fce7150b-d447-48c4-af9d-194c1821b192/volumes"
Jul 28 15:23:14.269557 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.269533    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7424-140452070997568"]
Jul 28 15:23:14.269885 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.269872    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fce7150b-d447-48c4-af9d-194c1821b192" containerName="main"
Jul 28 15:23:14.269924 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.269887    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fce7150b-d447-48c4-af9d-194c1821b192" containerName="main"
Jul 28 15:23:14.269924 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.269906    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be12b2fe-504f-4247-b371-cfc42318858e" containerName="main"
Jul 28 15:23:14.269924 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.269913    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be12b2fe-504f-4247-b371-cfc42318858e" containerName="main"
Jul 28 15:23:14.270021 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.269985    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fce7150b-d447-48c4-af9d-194c1821b192" containerName="main"
Jul 28 15:23:14.270021 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.269997    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be12b2fe-504f-4247-b371-cfc42318858e" containerName="main"
Jul 28 15:23:14.274367 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.274350    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:14.279826 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.279804    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ctx62\""
Jul 28 15:23:14.297867 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.297844    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7424-140452070997568"]
Jul 28 15:23:14.374768 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.374732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jchg5\" (UniqueName: \"kubernetes.io/projected/c8f52506-355c-4e12-823e-5d46f854f8b7-kube-api-access-jchg5\") pod \"test-curl-7424-140452070997568\" (UID: \"c8f52506-355c-4e12-823e-5d46f854f8b7\") " pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:14.475339 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.475265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jchg5\" (UniqueName: \"kubernetes.io/projected/c8f52506-355c-4e12-823e-5d46f854f8b7-kube-api-access-jchg5\") pod \"test-curl-7424-140452070997568\" (UID: \"c8f52506-355c-4e12-823e-5d46f854f8b7\") " pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:14.510437 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.510415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jchg5\" (UniqueName: \"kubernetes.io/projected/c8f52506-355c-4e12-823e-5d46f854f8b7-kube-api-access-jchg5\") pod \"test-curl-7424-140452070997568\" (UID: \"c8f52506-355c-4e12-823e-5d46f854f8b7\") " pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:14.583136 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.583113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:14.755841 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.755814    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7424-140452070997568"]
Jul 28 15:23:14.757897 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:23:14.757869    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8f52506_355c_4e12_823e_5d46f854f8b7.slice/crio-7c6f952b996846b453537a3ca1a99bf5c41c1654d50d7242476ac0b4b201e625 WatchSource:0}: Error finding container 7c6f952b996846b453537a3ca1a99bf5c41c1654d50d7242476ac0b4b201e625: Status 404 returned error can't find the container with id 7c6f952b996846b453537a3ca1a99bf5c41c1654d50d7242476ac0b4b201e625
Jul 28 15:23:14.758997 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:14.758981    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 15:23:15.755418 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:15.755325    2579 generic.go:356] "Generic (PLEG): container finished" podID="c8f52506-355c-4e12-823e-5d46f854f8b7" containerID="f08035cdd446239d9671a05e2b0340f4314de21aa4a215a3c524f9dc3ca03386" exitCode=0
Jul 28 15:23:15.755418 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:15.755402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7424-140452070997568" event={"ID":"c8f52506-355c-4e12-823e-5d46f854f8b7","Type":"ContainerDied","Data":"f08035cdd446239d9671a05e2b0340f4314de21aa4a215a3c524f9dc3ca03386"}
Jul 28 15:23:15.755832 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:15.755439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7424-140452070997568" event={"ID":"c8f52506-355c-4e12-823e-5d46f854f8b7","Type":"ContainerStarted","Data":"7c6f952b996846b453537a3ca1a99bf5c41c1654d50d7242476ac0b4b201e625"}
Jul 28 15:23:16.896400 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:16.896374    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:16.946490 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:16.946462    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7424-140452070997568_c8f52506-355c-4e12-823e-5d46f854f8b7/test-curl-7424-140452070997568/0.log"
Jul 28 15:23:16.981699 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:16.981666    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7424-140452070997568"]
Jul 28 15:23:16.984418 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:16.984385    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7424-140452070997568"]
Jul 28 15:23:17.000186 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:17.000164    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jchg5\" (UniqueName: \"kubernetes.io/projected/c8f52506-355c-4e12-823e-5d46f854f8b7-kube-api-access-jchg5\") pod \"c8f52506-355c-4e12-823e-5d46f854f8b7\" (UID: \"c8f52506-355c-4e12-823e-5d46f854f8b7\") "
Jul 28 15:23:17.002396 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:17.002371    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f52506-355c-4e12-823e-5d46f854f8b7-kube-api-access-jchg5" (OuterVolumeSpecName: "kube-api-access-jchg5") pod "c8f52506-355c-4e12-823e-5d46f854f8b7" (UID: "c8f52506-355c-4e12-823e-5d46f854f8b7"). InnerVolumeSpecName "kube-api-access-jchg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:23:17.101139 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:17.101051    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jchg5\" (UniqueName: \"kubernetes.io/projected/c8f52506-355c-4e12-823e-5d46f854f8b7-kube-api-access-jchg5\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:23:17.766513 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:17.766483    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:17.766513 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:17.766503    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c6f952b996846b453537a3ca1a99bf5c41c1654d50d7242476ac0b4b201e625"
Jul 28 15:23:17.777397 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:23:17.777370    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7424-140452070997568\" is forbidden: User \"system:node:ip-10-0-141-50.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-50.ec2.internal' and this object" podUID="c8f52506-355c-4e12-823e-5d46f854f8b7" pod="opendatahub/test-curl-7424-140452070997568"
Jul 28 15:23:18.194813 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:18.194780    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8f52506-355c-4e12-823e-5d46f854f8b7" path="/var/lib/kubelet/pods/c8f52506-355c-4e12-823e-5d46f854f8b7/volumes"
Jul 28 15:23:24.861414 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.861381    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"]
Jul 28 15:23:24.861787 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.861775    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8f52506-355c-4e12-823e-5d46f854f8b7" containerName="test-curl-7424-140452070997568"
Jul 28 15:23:24.861787 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.861788    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f52506-355c-4e12-823e-5d46f854f8b7" containerName="test-curl-7424-140452070997568"
Jul 28 15:23:24.861877 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.861848    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8f52506-355c-4e12-823e-5d46f854f8b7" containerName="test-curl-7424-140452070997568"
Jul 28 15:23:24.865125 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.865109    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:24.868315 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.868284    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-aa49c3-dockercfg-4fcgv\""
Jul 28 15:23:24.868769 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.868752    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-aa49c3\""
Jul 28 15:23:24.882100 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.882074    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"]
Jul 28 15:23:24.969759 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.969724    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94vzh\" (UniqueName: \"kubernetes.io/projected/c045a403-3008-4eec-81c5-12c4ca40da40-kube-api-access-94vzh\") pod \"payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:24.969759 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:24.969761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c045a403-3008-4eec-81c5-12c4ca40da40-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:25.070623 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:25.070584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c045a403-3008-4eec-81c5-12c4ca40da40-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:25.070808 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:25.070730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94vzh\" (UniqueName: \"kubernetes.io/projected/c045a403-3008-4eec-81c5-12c4ca40da40-kube-api-access-94vzh\") pod \"payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:25.071265 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:25.071243    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c045a403-3008-4eec-81c5-12c4ca40da40-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:25.083509 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:25.083484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94vzh\" (UniqueName: \"kubernetes.io/projected/c045a403-3008-4eec-81c5-12c4ca40da40-kube-api-access-94vzh\") pod \"payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:25.175712 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:25.175640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:25.314907 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:25.314880    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"]
Jul 28 15:23:25.316167 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:23:25.316138    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc045a403_3008_4eec_81c5_12c4ca40da40.slice/crio-ea7cb4ee85f22f7d8e9b2ef5df45cf0b1f2e1ca41abac77e8d36d6a029b13237 WatchSource:0}: Error finding container ea7cb4ee85f22f7d8e9b2ef5df45cf0b1f2e1ca41abac77e8d36d6a029b13237: Status 404 returned error can't find the container with id ea7cb4ee85f22f7d8e9b2ef5df45cf0b1f2e1ca41abac77e8d36d6a029b13237
Jul 28 15:23:25.806505 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:25.806467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx" event={"ID":"c045a403-3008-4eec-81c5-12c4ca40da40","Type":"ContainerStarted","Data":"ea7cb4ee85f22f7d8e9b2ef5df45cf0b1f2e1ca41abac77e8d36d6a029b13237"}
Jul 28 15:23:27.818025 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:27.818000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx" event={"ID":"c045a403-3008-4eec-81c5-12c4ca40da40","Type":"ContainerStarted","Data":"f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88"}
Jul 28 15:23:27.818423 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:27.818115    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:27.835517 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:27.835478    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx" podStartSLOduration=1.413817055 podStartE2EDuration="3.835463466s" totalImagesPullingTime="2.421646411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:23:27.835329707 +0000 UTC m=+3878.123190678" watchObservedRunningTime="2026-07-28 15:23:27.835463466 +0000 UTC m=+3878.123324408"
Jul 28 15:23:38.825043 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:38.825014    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:23:50.046481 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.046451    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"]
Jul 28 15:23:50.055430 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.055410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.058752 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.058727    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-034645-dockercfg-tgclw\""
Jul 28 15:23:50.058862 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.058837    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-034645\""
Jul 28 15:23:50.062997 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.062787    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"]
Jul 28 15:23:50.111116 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.111091    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mprcr\" (UniqueName: \"kubernetes.io/projected/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-kube-api-access-mprcr\") pod \"payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.111271 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.111171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.212514 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.212484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.212669 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.212542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprcr\" (UniqueName: \"kubernetes.io/projected/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-kube-api-access-mprcr\") pod \"payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.213135 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.213115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.222096 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.222075    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mprcr\" (UniqueName: \"kubernetes.io/projected/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-kube-api-access-mprcr\") pod \"payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.367013 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.366942    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.403196 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.403171    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:23:50.415183 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.415161    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:23:50.712194 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.712171    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"]
Jul 28 15:23:50.714197 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:23:50.714173    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a27e5f1_3401_403a_9ca1_0d30b6c4c743.slice/crio-219a97994b9928f94c85e7d1f4657e1d10a3c1e5b4177b761ea791ae9c03b57e WatchSource:0}: Error finding container 219a97994b9928f94c85e7d1f4657e1d10a3c1e5b4177b761ea791ae9c03b57e: Status 404 returned error can't find the container with id 219a97994b9928f94c85e7d1f4657e1d10a3c1e5b4177b761ea791ae9c03b57e
Jul 28 15:23:50.944805 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.944769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x" event={"ID":"4a27e5f1-3401-403a-9ca1-0d30b6c4c743","Type":"ContainerStarted","Data":"a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e"}
Jul 28 15:23:50.945002 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.944934    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:23:50.945002 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.944957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x" event={"ID":"4a27e5f1-3401-403a-9ca1-0d30b6c4c743","Type":"ContainerStarted","Data":"219a97994b9928f94c85e7d1f4657e1d10a3c1e5b4177b761ea791ae9c03b57e"}
Jul 28 15:23:50.966809 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:23:50.966711    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x" podStartSLOduration=0.966692063 podStartE2EDuration="966.692063ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:23:50.962835905 +0000 UTC m=+3901.250696867" watchObservedRunningTime="2026-07-28 15:23:50.966692063 +0000 UTC m=+3901.254553010"
Jul 28 15:24:01.952657 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:01.952627    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:24:13.418987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:13.418957    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x_4a27e5f1-3401-403a-9ca1-0d30b6c4c743/payload-processing/0.log"
Jul 28 15:24:17.254904 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.254870    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"]
Jul 28 15:24:17.258221 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.258196    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.273780 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.273754    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-aa49c3\"/\"kube-root-ca.crt\""
Jul 28 15:24:17.273948 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.273798    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-aa49c3\"/\"default-dockercfg-pxm67\""
Jul 28 15:24:17.273948 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.273754    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-aa49c3\"/\"openshift-service-ca.crt\""
Jul 28 15:24:17.284130 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.284111    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-aa49c3\"/\"ipp-route-aa49c3-kserve-self-signed-certs\""
Jul 28 15:24:17.301430 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.301409    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"]
Jul 28 15:24:17.371040 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.371010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-model-cache\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.371184 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.371056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-home\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.371184 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.371078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tmp-dir\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.371184 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.371145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tls-certs\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.371322 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.371200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-dshm\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.371322 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.371264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw5fh\" (UniqueName: \"kubernetes.io/projected/509fb0c5-9d18-40de-bc91-1b2236c8cee7-kube-api-access-bw5fh\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.472793 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.472757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw5fh\" (UniqueName: \"kubernetes.io/projected/509fb0c5-9d18-40de-bc91-1b2236c8cee7-kube-api-access-bw5fh\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.472937 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.472815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-model-cache\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.472937 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.472860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-home\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.472937 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.472886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tmp-dir\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.472937 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.472924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tls-certs\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.473159 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.472972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-dshm\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.473355 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.473324    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-model-cache\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.473355 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.473341    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tmp-dir\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.473514 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.473392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-home\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.475254 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.475228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-dshm\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.475567 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.475552    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tls-certs\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.494777 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.494751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw5fh\" (UniqueName: \"kubernetes.io/projected/509fb0c5-9d18-40de-bc91-1b2236c8cee7-kube-api-access-bw5fh\") pod \"ipp-route-aa49c3-kserve-f8c7c5847-wxzhj\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") " pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.567074 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.566998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:17.723467 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:17.723444    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"]
Jul 28 15:24:17.725799 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:24:17.725773    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod509fb0c5_9d18_40de_bc91_1b2236c8cee7.slice/crio-d62709190acddf0523b4f462c00886d52d1ab35bc093bca1612417cff24e0db8 WatchSource:0}: Error finding container d62709190acddf0523b4f462c00886d52d1ab35bc093bca1612417cff24e0db8: Status 404 returned error can't find the container with id d62709190acddf0523b4f462c00886d52d1ab35bc093bca1612417cff24e0db8
Jul 28 15:24:18.082920 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:18.082884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj" event={"ID":"509fb0c5-9d18-40de-bc91-1b2236c8cee7","Type":"ContainerStarted","Data":"6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2"}
Jul 28 15:24:18.082920 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:18.082926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj" event={"ID":"509fb0c5-9d18-40de-bc91-1b2236c8cee7","Type":"ContainerStarted","Data":"d62709190acddf0523b4f462c00886d52d1ab35bc093bca1612417cff24e0db8"}
Jul 28 15:24:18.123021 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:18.122965    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj" podStartSLOduration=1.122948445 podStartE2EDuration="1.122948445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:24:18.120695804 +0000 UTC m=+3928.408556747" watchObservedRunningTime="2026-07-28 15:24:18.122948445 +0000 UTC m=+3928.410809387"
Jul 28 15:24:27.567457 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:27.567419    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:27.567457 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:27.567467    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:27.579917 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:27.579892    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:28.142067 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:28.142036    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:24:58.665673 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:24:58.665590    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x_4a27e5f1-3401-403a-9ca1-0d30b6c4c743/payload-processing/0.log"
Jul 28 15:25:07.399361 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.399328    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"]
Jul 28 15:25:07.403031 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.403013    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.408687 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.408668    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-9498a6\""
Jul 28 15:25:07.408687 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.408671    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-9498a6-dockercfg-62jnd\""
Jul 28 15:25:07.419004 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.418979    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"]
Jul 28 15:25:07.419411 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.419384    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85c7427d-6b22-4262-bb39-47dcffa3f01c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.419519 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.419417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvk8\" (UniqueName: \"kubernetes.io/projected/85c7427d-6b22-4262-bb39-47dcffa3f01c-kube-api-access-7cvk8\") pod \"payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.482077 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.482048    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"]
Jul 28 15:25:07.485435 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.485421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.498015 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.497994    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"]
Jul 28 15:25:07.520768 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.520743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85c7427d-6b22-4262-bb39-47dcffa3f01c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.520907 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.520770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvk8\" (UniqueName: \"kubernetes.io/projected/85c7427d-6b22-4262-bb39-47dcffa3f01c-kube-api-access-7cvk8\") pod \"payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.520907 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.520803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.520907 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.520830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5twg\" (UniqueName: \"kubernetes.io/projected/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-kube-api-access-f5twg\") pod \"payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.521507 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.521486    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85c7427d-6b22-4262-bb39-47dcffa3f01c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.530139 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.530111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvk8\" (UniqueName: \"kubernetes.io/projected/85c7427d-6b22-4262-bb39-47dcffa3f01c-kube-api-access-7cvk8\") pod \"payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.621449 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.621419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.621590 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.621457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5twg\" (UniqueName: \"kubernetes.io/projected/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-kube-api-access-f5twg\") pod \"payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.622069 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.622050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.632031 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.632007    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5twg\" (UniqueName: \"kubernetes.io/projected/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-kube-api-access-f5twg\") pod \"payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.712753 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.712734    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:07.795045 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.794819    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:07.859670 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.859573    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"]
Jul 28 15:25:07.860711 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:25:07.860683    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c7427d_6b22_4262_bb39_47dcffa3f01c.slice/crio-283707c0b1c44fbfae299a2993a4061d1f0c1c1dc5c8e8771b918048d34b6bea WatchSource:0}: Error finding container 283707c0b1c44fbfae299a2993a4061d1f0c1c1dc5c8e8771b918048d34b6bea: Status 404 returned error can't find the container with id 283707c0b1c44fbfae299a2993a4061d1f0c1c1dc5c8e8771b918048d34b6bea
Jul 28 15:25:07.943932 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:07.943905    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"]
Jul 28 15:25:07.944544 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:25:07.944521    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e58adb6_dd5a_4f1d_a957_5e392331d1bc.slice/crio-d0641a3915560b5c3e592b0aa300c9c66023bc56d6d8719f47603dcf1c1d5070 WatchSource:0}: Error finding container d0641a3915560b5c3e592b0aa300c9c66023bc56d6d8719f47603dcf1c1d5070: Status 404 returned error can't find the container with id d0641a3915560b5c3e592b0aa300c9c66023bc56d6d8719f47603dcf1c1d5070
Jul 28 15:25:08.340153 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.340064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf" event={"ID":"85c7427d-6b22-4262-bb39-47dcffa3f01c","Type":"ContainerStarted","Data":"970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c"}
Jul 28 15:25:08.340153 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.340123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf" event={"ID":"85c7427d-6b22-4262-bb39-47dcffa3f01c","Type":"ContainerStarted","Data":"283707c0b1c44fbfae299a2993a4061d1f0c1c1dc5c8e8771b918048d34b6bea"}
Jul 28 15:25:08.340548 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.340529    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:08.341877 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.341854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7" event={"ID":"3e58adb6-dd5a-4f1d-a957-5e392331d1bc","Type":"ContainerStarted","Data":"f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971"}
Jul 28 15:25:08.341987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.341900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7" event={"ID":"3e58adb6-dd5a-4f1d-a957-5e392331d1bc","Type":"ContainerStarted","Data":"d0641a3915560b5c3e592b0aa300c9c66023bc56d6d8719f47603dcf1c1d5070"}
Jul 28 15:25:08.341987 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.341928    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:08.372415 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.372373    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf" podStartSLOduration=1.372361742 podStartE2EDuration="1.372361742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:25:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:25:08.37018508 +0000 UTC m=+3978.658046026" watchObservedRunningTime="2026-07-28 15:25:08.372361742 +0000 UTC m=+3978.660222686"
Jul 28 15:25:08.401176 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:08.401138    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7" podStartSLOduration=1.401125655 podStartE2EDuration="1.401125655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:25:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:25:08.398548351 +0000 UTC m=+3978.686409333" watchObservedRunningTime="2026-07-28 15:25:08.401125655 +0000 UTC m=+3978.688986607"
Jul 28 15:25:19.348092 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:19.348045    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf" podUID="85c7427d-6b22-4262-bb39-47dcffa3f01c" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.52:9004: connect: connection refused"
Jul 28 15:25:19.348847 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:19.348611    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:29.348341 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:29.348307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:33.432984 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.432950    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"]
Jul 28 15:25:33.437012 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.436996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"
Jul 28 15:25:33.442182 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.442163    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cbpmp\""
Jul 28 15:25:33.442518 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.442502    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 15:25:33.458262 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.458233    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"]
Jul 28 15:25:33.560536 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.560505    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa1776bf-8155-453f-a588-b6d3e52b169e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp\" (UID: \"aa1776bf-8155-453f-a588-b6d3e52b169e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"
Jul 28 15:25:33.661380 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.661345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa1776bf-8155-453f-a588-b6d3e52b169e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp\" (UID: \"aa1776bf-8155-453f-a588-b6d3e52b169e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"
Jul 28 15:25:33.661949 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.661931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa1776bf-8155-453f-a588-b6d3e52b169e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp\" (UID: \"aa1776bf-8155-453f-a588-b6d3e52b169e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"
Jul 28 15:25:33.748400 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.748332    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"
Jul 28 15:25:33.907205 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:33.907174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"]
Jul 28 15:25:33.909994 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:25:33.909959    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa1776bf_8155_453f_a588_b6d3e52b169e.slice/crio-9fdbbcc3a3ca18219db078914731b18d60b7a4206f41bcf20ea9cd3ae3f11e85 WatchSource:0}: Error finding container 9fdbbcc3a3ca18219db078914731b18d60b7a4206f41bcf20ea9cd3ae3f11e85: Status 404 returned error can't find the container with id 9fdbbcc3a3ca18219db078914731b18d60b7a4206f41bcf20ea9cd3ae3f11e85
Jul 28 15:25:34.477389 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:34.477357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp" event={"ID":"aa1776bf-8155-453f-a588-b6d3e52b169e","Type":"ContainerStarted","Data":"9fdbbcc3a3ca18219db078914731b18d60b7a4206f41bcf20ea9cd3ae3f11e85"}
Jul 28 15:25:36.489134 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:36.489037    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp" event={"ID":"aa1776bf-8155-453f-a588-b6d3e52b169e","Type":"ContainerStarted","Data":"2200bb7c104d3ffd5b4c51c89429de5b2b6bc91858b23dfc93aa3ce10d3385c2"}
Jul 28 15:25:36.532943 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:36.532898    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp" podStartSLOduration=1.526311684 podStartE2EDuration="3.532882375s" totalImagesPullingTime="2.006570691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:25:36.531955547 +0000 UTC m=+4006.819816491" watchObservedRunningTime="2026-07-28 15:25:36.532882375 +0000 UTC m=+4006.820743318"
Jul 28 15:25:37.495921 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:37.495881    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa1776bf-8155-453f-a588-b6d3e52b169e" containerID="2200bb7c104d3ffd5b4c51c89429de5b2b6bc91858b23dfc93aa3ce10d3385c2" exitCode=0
Jul 28 15:25:37.496313 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:37.495948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp" event={"ID":"aa1776bf-8155-453f-a588-b6d3e52b169e","Type":"ContainerDied","Data":"2200bb7c104d3ffd5b4c51c89429de5b2b6bc91858b23dfc93aa3ce10d3385c2"}
Jul 28 15:25:38.633872 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:38.633847    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"
Jul 28 15:25:38.809609 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:38.809527    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa1776bf-8155-453f-a588-b6d3e52b169e-maas-api-service-ca\") pod \"aa1776bf-8155-453f-a588-b6d3e52b169e\" (UID: \"aa1776bf-8155-453f-a588-b6d3e52b169e\") "
Jul 28 15:25:38.809983 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:38.809956    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa1776bf-8155-453f-a588-b6d3e52b169e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa1776bf-8155-453f-a588-b6d3e52b169e" (UID: "aa1776bf-8155-453f-a588-b6d3e52b169e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:25:38.910790 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:38.910759    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa1776bf-8155-453f-a588-b6d3e52b169e-maas-api-service-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:25:39.505977 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:39.505940    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp" event={"ID":"aa1776bf-8155-453f-a588-b6d3e52b169e","Type":"ContainerDied","Data":"9fdbbcc3a3ca18219db078914731b18d60b7a4206f41bcf20ea9cd3ae3f11e85"}
Jul 28 15:25:39.505977 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:39.505962    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"
Jul 28 15:25:39.506180 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:39.505986    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fdbbcc3a3ca18219db078914731b18d60b7a4206f41bcf20ea9cd3ae3f11e85"
Jul 28 15:25:45.103945 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.103911    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"]
Jul 28 15:25:45.104440 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.104161    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7" podUID="3e58adb6-dd5a-4f1d-a957-5e392331d1bc" containerName="payload-processing" containerID="cri-o://f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971" gracePeriod=30
Jul 28 15:25:45.112188 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.112164    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"]
Jul 28 15:25:45.112428 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.112384    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf" podUID="85c7427d-6b22-4262-bb39-47dcffa3f01c" containerName="payload-pre-processing" containerID="cri-o://970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c" gracePeriod=30
Jul 28 15:25:45.255573 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.255539    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"]
Jul 28 15:25:45.259024 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.258996    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9498a6-k94rp"]
Jul 28 15:25:45.393443 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.393418    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:45.395484 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.395466    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:45.475720 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.475696    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-plugins-config-volume\") pod \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") "
Jul 28 15:25:45.475883 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.475734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85c7427d-6b22-4262-bb39-47dcffa3f01c-plugins-config-volume\") pod \"85c7427d-6b22-4262-bb39-47dcffa3f01c\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") "
Jul 28 15:25:45.475883 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.475791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5twg\" (UniqueName: \"kubernetes.io/projected/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-kube-api-access-f5twg\") pod \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\" (UID: \"3e58adb6-dd5a-4f1d-a957-5e392331d1bc\") "
Jul 28 15:25:45.475883 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.475831    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cvk8\" (UniqueName: \"kubernetes.io/projected/85c7427d-6b22-4262-bb39-47dcffa3f01c-kube-api-access-7cvk8\") pod \"85c7427d-6b22-4262-bb39-47dcffa3f01c\" (UID: \"85c7427d-6b22-4262-bb39-47dcffa3f01c\") "
Jul 28 15:25:45.476129 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.476105    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e58adb6-dd5a-4f1d-a957-5e392331d1bc" (UID: "3e58adb6-dd5a-4f1d-a957-5e392331d1bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:25:45.476204 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.476121    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85c7427d-6b22-4262-bb39-47dcffa3f01c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85c7427d-6b22-4262-bb39-47dcffa3f01c" (UID: "85c7427d-6b22-4262-bb39-47dcffa3f01c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:25:45.476300 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.476206    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-plugins-config-volume\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:25:45.476300 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.476251    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85c7427d-6b22-4262-bb39-47dcffa3f01c-plugins-config-volume\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:25:45.478025 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.477998    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-kube-api-access-f5twg" (OuterVolumeSpecName: "kube-api-access-f5twg") pod "3e58adb6-dd5a-4f1d-a957-5e392331d1bc" (UID: "3e58adb6-dd5a-4f1d-a957-5e392331d1bc"). InnerVolumeSpecName "kube-api-access-f5twg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:25:45.478114 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.478032    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85c7427d-6b22-4262-bb39-47dcffa3f01c-kube-api-access-7cvk8" (OuterVolumeSpecName: "kube-api-access-7cvk8") pod "85c7427d-6b22-4262-bb39-47dcffa3f01c" (UID: "85c7427d-6b22-4262-bb39-47dcffa3f01c"). InnerVolumeSpecName "kube-api-access-7cvk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:25:45.537430 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.537398    2579 generic.go:356] "Generic (PLEG): container finished" podID="85c7427d-6b22-4262-bb39-47dcffa3f01c" containerID="970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c" exitCode=0
Jul 28 15:25:45.537550 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.537447    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"
Jul 28 15:25:45.537550 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.537486    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf" event={"ID":"85c7427d-6b22-4262-bb39-47dcffa3f01c","Type":"ContainerDied","Data":"970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c"}
Jul 28 15:25:45.537671 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.537578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf" event={"ID":"85c7427d-6b22-4262-bb39-47dcffa3f01c","Type":"ContainerDied","Data":"283707c0b1c44fbfae299a2993a4061d1f0c1c1dc5c8e8771b918048d34b6bea"}
Jul 28 15:25:45.537671 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.537612    2579 scope.go:117] "RemoveContainer" containerID="970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c"
Jul 28 15:25:45.538996 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.538970    2579 generic.go:356] "Generic (PLEG): container finished" podID="3e58adb6-dd5a-4f1d-a957-5e392331d1bc" containerID="f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971" exitCode=0
Jul 28 15:25:45.539098 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.539015    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"
Jul 28 15:25:45.539155 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.539011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7" event={"ID":"3e58adb6-dd5a-4f1d-a957-5e392331d1bc","Type":"ContainerDied","Data":"f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971"}
Jul 28 15:25:45.539226 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.539159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7" event={"ID":"3e58adb6-dd5a-4f1d-a957-5e392331d1bc","Type":"ContainerDied","Data":"d0641a3915560b5c3e592b0aa300c9c66023bc56d6d8719f47603dcf1c1d5070"}
Jul 28 15:25:45.550900 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.550885    2579 scope.go:117] "RemoveContainer" containerID="970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c"
Jul 28 15:25:45.551143 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:25:45.551126    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c\": container with ID starting with 970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c not found: ID does not exist" containerID="970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c"
Jul 28 15:25:45.551187 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.551150    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c"} err="failed to get container status \"970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c\": rpc error: code = NotFound desc = could not find container \"970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c\": container with ID starting with 970858b0fc99035a0d42bece810c2b55e4de1564010e26d71d37896afbc5203c not found: ID does not exist"
Jul 28 15:25:45.551187 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.551172    2579 scope.go:117] "RemoveContainer" containerID="f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971"
Jul 28 15:25:45.558576 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.558561    2579 scope.go:117] "RemoveContainer" containerID="f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971"
Jul 28 15:25:45.558795 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:25:45.558779    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971\": container with ID starting with f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971 not found: ID does not exist" containerID="f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971"
Jul 28 15:25:45.558848 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.558800    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971"} err="failed to get container status \"f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971\": rpc error: code = NotFound desc = could not find container \"f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971\": container with ID starting with f61928892dc76267886dce93ae8acdbdab9fb58391bdec761750a9a32c96c971 not found: ID does not exist"
Jul 28 15:25:45.576649 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.576620    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5twg\" (UniqueName: \"kubernetes.io/projected/3e58adb6-dd5a-4f1d-a957-5e392331d1bc-kube-api-access-f5twg\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:25:45.576649 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.576643    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cvk8\" (UniqueName: \"kubernetes.io/projected/85c7427d-6b22-4262-bb39-47dcffa3f01c-kube-api-access-7cvk8\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:25:45.580680 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.580661    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"]
Jul 28 15:25:45.587390 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.587371    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9498a6-7bff668956-j5wpf"]
Jul 28 15:25:45.610487 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.610468    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"]
Jul 28 15:25:45.624235 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:45.624162    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9498a6-6d44f964c8-md2x7"]
Jul 28 15:25:46.192672 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:46.192643    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e58adb6-dd5a-4f1d-a957-5e392331d1bc" path="/var/lib/kubelet/pods/3e58adb6-dd5a-4f1d-a957-5e392331d1bc/volumes"
Jul 28 15:25:46.193037 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:46.193003    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85c7427d-6b22-4262-bb39-47dcffa3f01c" path="/var/lib/kubelet/pods/85c7427d-6b22-4262-bb39-47dcffa3f01c/volumes"
Jul 28 15:25:46.193313 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:46.193300    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa1776bf-8155-453f-a588-b6d3e52b169e" path="/var/lib/kubelet/pods/aa1776bf-8155-453f-a588-b6d3e52b169e/volumes"
Jul 28 15:25:50.572089 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:50.572058    2579 scope.go:117] "RemoveContainer" containerID="e0c257174f276d54c3105d999ad2473a9dd17197652bc2dbe401ede45770db11"
Jul 28 15:25:56.888323 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888282    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"]
Jul 28 15:25:56.888715 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888674    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85c7427d-6b22-4262-bb39-47dcffa3f01c" containerName="payload-pre-processing"
Jul 28 15:25:56.888715 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888685    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="85c7427d-6b22-4262-bb39-47dcffa3f01c" containerName="payload-pre-processing"
Jul 28 15:25:56.888715 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888693    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e58adb6-dd5a-4f1d-a957-5e392331d1bc" containerName="payload-processing"
Jul 28 15:25:56.888715 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888698    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e58adb6-dd5a-4f1d-a957-5e392331d1bc" containerName="payload-processing"
Jul 28 15:25:56.888715 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888705    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa1776bf-8155-453f-a588-b6d3e52b169e" containerName="revoke-keys"
Jul 28 15:25:56.888884 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888711    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa1776bf-8155-453f-a588-b6d3e52b169e" containerName="revoke-keys"
Jul 28 15:25:56.888884 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888793    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e58adb6-dd5a-4f1d-a957-5e392331d1bc" containerName="payload-processing"
Jul 28 15:25:56.888884 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888802    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="85c7427d-6b22-4262-bb39-47dcffa3f01c" containerName="payload-pre-processing"
Jul 28 15:25:56.888884 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.888810    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa1776bf-8155-453f-a588-b6d3e52b169e" containerName="revoke-keys"
Jul 28 15:25:56.893477 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.893461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"
Jul 28 15:25:56.896754 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.896727    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cbpmp\""
Jul 28 15:25:56.898029 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.898009    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 15:25:56.899361 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.899340    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"]
Jul 28 15:25:56.980137 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:56.980106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9862e7-577d-4209-af9b-3f2ea2707e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b\" (UID: \"1a9862e7-577d-4209-af9b-3f2ea2707e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"
Jul 28 15:25:57.081270 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:57.081235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9862e7-577d-4209-af9b-3f2ea2707e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b\" (UID: \"1a9862e7-577d-4209-af9b-3f2ea2707e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"
Jul 28 15:25:57.081821 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:57.081800    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9862e7-577d-4209-af9b-3f2ea2707e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b\" (UID: \"1a9862e7-577d-4209-af9b-3f2ea2707e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"
Jul 28 15:25:57.205340 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:57.205300    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"
Jul 28 15:25:57.334997 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:57.334973    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"]
Jul 28 15:25:57.337120 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:25:57.337095    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9862e7_577d_4209_af9b_3f2ea2707e36.slice/crio-97735fb9f48713d43fc0c5d1c7d033e181455a1aa38d283dcdc8d55a3ae10a4a WatchSource:0}: Error finding container 97735fb9f48713d43fc0c5d1c7d033e181455a1aa38d283dcdc8d55a3ae10a4a: Status 404 returned error can't find the container with id 97735fb9f48713d43fc0c5d1c7d033e181455a1aa38d283dcdc8d55a3ae10a4a
Jul 28 15:25:57.602959 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:57.602930    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b" event={"ID":"1a9862e7-577d-4209-af9b-3f2ea2707e36","Type":"ContainerStarted","Data":"97735fb9f48713d43fc0c5d1c7d033e181455a1aa38d283dcdc8d55a3ae10a4a"}
Jul 28 15:25:58.609107 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:58.609072    2579 generic.go:356] "Generic (PLEG): container finished" podID="1a9862e7-577d-4209-af9b-3f2ea2707e36" containerID="1d9ef3b11c1073e7920a5ba8bceb16b516664de725657ca43ff1c7b926205095" exitCode=0
Jul 28 15:25:58.609555 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:58.609162    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b" event={"ID":"1a9862e7-577d-4209-af9b-3f2ea2707e36","Type":"ContainerDied","Data":"1d9ef3b11c1073e7920a5ba8bceb16b516664de725657ca43ff1c7b926205095"}
Jul 28 15:25:59.742097 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:59.742075    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"
Jul 28 15:25:59.805908 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:59.805869    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9862e7-577d-4209-af9b-3f2ea2707e36-maas-api-service-ca\") pod \"1a9862e7-577d-4209-af9b-3f2ea2707e36\" (UID: \"1a9862e7-577d-4209-af9b-3f2ea2707e36\") "
Jul 28 15:25:59.806239 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:59.806197    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a9862e7-577d-4209-af9b-3f2ea2707e36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a9862e7-577d-4209-af9b-3f2ea2707e36" (UID: "1a9862e7-577d-4209-af9b-3f2ea2707e36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:25:59.906879 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:25:59.906791    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a9862e7-577d-4209-af9b-3f2ea2707e36-maas-api-service-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:00.620513 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:00.620430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b" event={"ID":"1a9862e7-577d-4209-af9b-3f2ea2707e36","Type":"ContainerDied","Data":"97735fb9f48713d43fc0c5d1c7d033e181455a1aa38d283dcdc8d55a3ae10a4a"}
Jul 28 15:26:00.620513 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:00.620481    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97735fb9f48713d43fc0c5d1c7d033e181455a1aa38d283dcdc8d55a3ae10a4a"
Jul 28 15:26:00.620513 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:00.620449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"
Jul 28 15:26:13.672907 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:13.672821    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"]
Jul 28 15:26:13.673458 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:13.673086    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx" podUID="c045a403-3008-4eec-81c5-12c4ca40da40" containerName="payload-pre-processing" containerID="cri-o://f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88" gracePeriod=30
Jul 28 15:26:13.877235 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:13.877183    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"]
Jul 28 15:26:13.879143 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:13.879117    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aa49c3-4f85b"]
Jul 28 15:26:13.933689 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:13.933662    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:26:14.038720 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.038694    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c045a403-3008-4eec-81c5-12c4ca40da40-plugins-config-volume\") pod \"c045a403-3008-4eec-81c5-12c4ca40da40\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") "
Jul 28 15:26:14.038906 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.038743    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94vzh\" (UniqueName: \"kubernetes.io/projected/c045a403-3008-4eec-81c5-12c4ca40da40-kube-api-access-94vzh\") pod \"c045a403-3008-4eec-81c5-12c4ca40da40\" (UID: \"c045a403-3008-4eec-81c5-12c4ca40da40\") "
Jul 28 15:26:14.039179 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.039150    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c045a403-3008-4eec-81c5-12c4ca40da40-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c045a403-3008-4eec-81c5-12c4ca40da40" (UID: "c045a403-3008-4eec-81c5-12c4ca40da40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:26:14.040868 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.040843    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c045a403-3008-4eec-81c5-12c4ca40da40-kube-api-access-94vzh" (OuterVolumeSpecName: "kube-api-access-94vzh") pod "c045a403-3008-4eec-81c5-12c4ca40da40" (UID: "c045a403-3008-4eec-81c5-12c4ca40da40"). InnerVolumeSpecName "kube-api-access-94vzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:26:14.140268 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.140236    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c045a403-3008-4eec-81c5-12c4ca40da40-plugins-config-volume\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:14.140268 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.140261    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94vzh\" (UniqueName: \"kubernetes.io/projected/c045a403-3008-4eec-81c5-12c4ca40da40-kube-api-access-94vzh\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:14.192990 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.192914    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a9862e7-577d-4209-af9b-3f2ea2707e36" path="/var/lib/kubelet/pods/1a9862e7-577d-4209-af9b-3f2ea2707e36/volumes"
Jul 28 15:26:14.701988 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.701948    2579 generic.go:356] "Generic (PLEG): container finished" podID="c045a403-3008-4eec-81c5-12c4ca40da40" containerID="f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88" exitCode=0
Jul 28 15:26:14.702435 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.701997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx" event={"ID":"c045a403-3008-4eec-81c5-12c4ca40da40","Type":"ContainerDied","Data":"f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88"}
Jul 28 15:26:14.702435 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.702007    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"
Jul 28 15:26:14.702435 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.702074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx" event={"ID":"c045a403-3008-4eec-81c5-12c4ca40da40","Type":"ContainerDied","Data":"ea7cb4ee85f22f7d8e9b2ef5df45cf0b1f2e1ca41abac77e8d36d6a029b13237"}
Jul 28 15:26:14.702435 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.702104    2579 scope.go:117] "RemoveContainer" containerID="f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88"
Jul 28 15:26:14.710746 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.710727    2579 scope.go:117] "RemoveContainer" containerID="f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88"
Jul 28 15:26:14.711006 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:26:14.710981    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88\": container with ID starting with f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88 not found: ID does not exist" containerID="f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88"
Jul 28 15:26:14.711111 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.711010    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88"} err="failed to get container status \"f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88\": rpc error: code = NotFound desc = could not find container \"f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88\": container with ID starting with f8268b0533e4d565997dad0502086d8f6287ffe3b364eaa6a68adc6d9e311d88 not found: ID does not exist"
Jul 28 15:26:14.721438 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.721417    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"]
Jul 28 15:26:14.725444 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:14.725422    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aa49c3-6d748f7cd8-2wscx"]
Jul 28 15:26:16.193027 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:16.192993    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c045a403-3008-4eec-81c5-12c4ca40da40" path="/var/lib/kubelet/pods/c045a403-3008-4eec-81c5-12c4ca40da40/volumes"
Jul 28 15:26:19.245999 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:19.245963    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"]
Jul 28 15:26:34.264761 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.264704    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj" podUID="509fb0c5-9d18-40de-bc91-1b2236c8cee7" containerName="main" containerID="cri-o://6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2" gracePeriod=45
Jul 28 15:26:34.517717 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.517657    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:26:34.529821 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.529801    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tmp-dir\") pod \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") "
Jul 28 15:26:34.529933 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.529836    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw5fh\" (UniqueName: \"kubernetes.io/projected/509fb0c5-9d18-40de-bc91-1b2236c8cee7-kube-api-access-bw5fh\") pod \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") "
Jul 28 15:26:34.529933 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.529875    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tls-certs\") pod \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") "
Jul 28 15:26:34.529933 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.529914    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-home\") pod \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") "
Jul 28 15:26:34.530102 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.529999    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-model-cache\") pod \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") "
Jul 28 15:26:34.530102 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.530045    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-dshm\") pod \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\" (UID: \"509fb0c5-9d18-40de-bc91-1b2236c8cee7\") "
Jul 28 15:26:34.530102 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.530069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "509fb0c5-9d18-40de-bc91-1b2236c8cee7" (UID: "509fb0c5-9d18-40de-bc91-1b2236c8cee7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:26:34.530292 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.530277    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-home" (OuterVolumeSpecName: "home") pod "509fb0c5-9d18-40de-bc91-1b2236c8cee7" (UID: "509fb0c5-9d18-40de-bc91-1b2236c8cee7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:26:34.530353 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.530310    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-model-cache" (OuterVolumeSpecName: "model-cache") pod "509fb0c5-9d18-40de-bc91-1b2236c8cee7" (UID: "509fb0c5-9d18-40de-bc91-1b2236c8cee7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:26:34.530410 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.530380    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tmp-dir\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:34.530410 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.530399    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-home\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:34.530514 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.530411    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-model-cache\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:34.532056 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.532030    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-dshm" (OuterVolumeSpecName: "dshm") pod "509fb0c5-9d18-40de-bc91-1b2236c8cee7" (UID: "509fb0c5-9d18-40de-bc91-1b2236c8cee7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 15:26:34.532151 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.532113    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "509fb0c5-9d18-40de-bc91-1b2236c8cee7" (UID: "509fb0c5-9d18-40de-bc91-1b2236c8cee7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 15:26:34.532248 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.532207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509fb0c5-9d18-40de-bc91-1b2236c8cee7-kube-api-access-bw5fh" (OuterVolumeSpecName: "kube-api-access-bw5fh") pod "509fb0c5-9d18-40de-bc91-1b2236c8cee7" (UID: "509fb0c5-9d18-40de-bc91-1b2236c8cee7"). InnerVolumeSpecName "kube-api-access-bw5fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:26:34.630983 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.630951    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509fb0c5-9d18-40de-bc91-1b2236c8cee7-dshm\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:34.630983 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.630977    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw5fh\" (UniqueName: \"kubernetes.io/projected/509fb0c5-9d18-40de-bc91-1b2236c8cee7-kube-api-access-bw5fh\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:34.630983 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.630986    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509fb0c5-9d18-40de-bc91-1b2236c8cee7-tls-certs\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:26:34.801592 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.801507    2579 generic.go:356] "Generic (PLEG): container finished" podID="509fb0c5-9d18-40de-bc91-1b2236c8cee7" containerID="6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2" exitCode=0
Jul 28 15:26:34.801592 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.801554    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"
Jul 28 15:26:34.801592 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.801566    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj" event={"ID":"509fb0c5-9d18-40de-bc91-1b2236c8cee7","Type":"ContainerDied","Data":"6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2"}
Jul 28 15:26:34.801810 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.801603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj" event={"ID":"509fb0c5-9d18-40de-bc91-1b2236c8cee7","Type":"ContainerDied","Data":"d62709190acddf0523b4f462c00886d52d1ab35bc093bca1612417cff24e0db8"}
Jul 28 15:26:34.801810 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.801627    2579 scope.go:117] "RemoveContainer" containerID="6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2"
Jul 28 15:26:34.810714 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.810696    2579 scope.go:117] "RemoveContainer" containerID="6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2"
Jul 28 15:26:34.810995 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:26:34.810976    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2\": container with ID starting with 6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2 not found: ID does not exist" containerID="6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2"
Jul 28 15:26:34.811059 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.811002    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2"} err="failed to get container status \"6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2\": rpc error: code = NotFound desc = could not find container \"6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2\": container with ID starting with 6932a62ba31324314ca3f9009f59583e22bebfd3424e181d930b129936a84cf2 not found: ID does not exist"
Jul 28 15:26:34.828199 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.828177    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"]
Jul 28 15:26:34.838182 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:34.838162    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-aa49c3/ipp-route-aa49c3-kserve-f8c7c5847-wxzhj"]
Jul 28 15:26:36.192076 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:26:36.192042    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="509fb0c5-9d18-40de-bc91-1b2236c8cee7" path="/var/lib/kubelet/pods/509fb0c5-9d18-40de-bc91-1b2236c8cee7/volumes"
Jul 28 15:27:04.021780 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.021751    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"]
Jul 28 15:27:04.022177 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022166    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9862e7-577d-4209-af9b-3f2ea2707e36" containerName="revoke-keys"
Jul 28 15:27:04.022237 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022179    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9862e7-577d-4209-af9b-3f2ea2707e36" containerName="revoke-keys"
Jul 28 15:27:04.022237 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022194    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509fb0c5-9d18-40de-bc91-1b2236c8cee7" containerName="main"
Jul 28 15:27:04.022237 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022200    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="509fb0c5-9d18-40de-bc91-1b2236c8cee7" containerName="main"
Jul 28 15:27:04.022237 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022207    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c045a403-3008-4eec-81c5-12c4ca40da40" containerName="payload-pre-processing"
Jul 28 15:27:04.022237 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022224    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c045a403-3008-4eec-81c5-12c4ca40da40" containerName="payload-pre-processing"
Jul 28 15:27:04.022401 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022291    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c045a403-3008-4eec-81c5-12c4ca40da40" containerName="payload-pre-processing"
Jul 28 15:27:04.022401 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022301    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="509fb0c5-9d18-40de-bc91-1b2236c8cee7" containerName="main"
Jul 28 15:27:04.022401 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.022306    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a9862e7-577d-4209-af9b-3f2ea2707e36" containerName="revoke-keys"
Jul 28 15:27:04.026753 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.026738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"
Jul 28 15:27:04.031011 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.030981    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cbpmp\""
Jul 28 15:27:04.031148 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.031101    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 15:27:04.040922 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.040897    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"]
Jul 28 15:27:04.204249 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.204196    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad253b-6525-421f-b9fb-004a34a1bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc\" (UID: \"54ad253b-6525-421f-b9fb-004a34a1bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"
Jul 28 15:27:04.305240 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.305139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad253b-6525-421f-b9fb-004a34a1bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc\" (UID: \"54ad253b-6525-421f-b9fb-004a34a1bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"
Jul 28 15:27:04.305800 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.305781    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad253b-6525-421f-b9fb-004a34a1bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc\" (UID: \"54ad253b-6525-421f-b9fb-004a34a1bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"
Jul 28 15:27:04.339842 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.339809    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"
Jul 28 15:27:04.471400 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.471376    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"]
Jul 28 15:27:04.473830 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:27:04.473803    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ad253b_6525_421f_b9fb_004a34a1bd01.slice/crio-79335509b0aceb1d41cd694799a499e1da84d2edf14608c5c6b0270047512216 WatchSource:0}: Error finding container 79335509b0aceb1d41cd694799a499e1da84d2edf14608c5c6b0270047512216: Status 404 returned error can't find the container with id 79335509b0aceb1d41cd694799a499e1da84d2edf14608c5c6b0270047512216
Jul 28 15:27:04.959457 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.959423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc" event={"ID":"54ad253b-6525-421f-b9fb-004a34a1bd01","Type":"ContainerStarted","Data":"3e75900944596338c695c23081900cb020222245c5503878383a9651e8c74508"}
Jul 28 15:27:04.959648 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.959478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc" event={"ID":"54ad253b-6525-421f-b9fb-004a34a1bd01","Type":"ContainerStarted","Data":"79335509b0aceb1d41cd694799a499e1da84d2edf14608c5c6b0270047512216"}
Jul 28 15:27:04.977241 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:04.977175    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc" podStartSLOduration=1.977160992 podStartE2EDuration="1.977160992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:27:04.975691533 +0000 UTC m=+4095.263552492" watchObservedRunningTime="2026-07-28 15:27:04.977160992 +0000 UTC m=+4095.265021934"
Jul 28 15:27:05.965847 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:05.965814    2579 generic.go:356] "Generic (PLEG): container finished" podID="54ad253b-6525-421f-b9fb-004a34a1bd01" containerID="3e75900944596338c695c23081900cb020222245c5503878383a9651e8c74508" exitCode=0
Jul 28 15:27:05.966225 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:05.965894    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc" event={"ID":"54ad253b-6525-421f-b9fb-004a34a1bd01","Type":"ContainerDied","Data":"3e75900944596338c695c23081900cb020222245c5503878383a9651e8c74508"}
Jul 28 15:27:07.096124 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:07.096102    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"
Jul 28 15:27:07.133356 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:07.133330    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad253b-6525-421f-b9fb-004a34a1bd01-maas-api-service-ca\") pod \"54ad253b-6525-421f-b9fb-004a34a1bd01\" (UID: \"54ad253b-6525-421f-b9fb-004a34a1bd01\") "
Jul 28 15:27:07.133631 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:07.133610    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54ad253b-6525-421f-b9fb-004a34a1bd01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54ad253b-6525-421f-b9fb-004a34a1bd01" (UID: "54ad253b-6525-421f-b9fb-004a34a1bd01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:07.235182 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:07.234999    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54ad253b-6525-421f-b9fb-004a34a1bd01-maas-api-service-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:27:07.978719 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:07.978694    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"
Jul 28 15:27:07.978719 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:07.978712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc" event={"ID":"54ad253b-6525-421f-b9fb-004a34a1bd01","Type":"ContainerDied","Data":"79335509b0aceb1d41cd694799a499e1da84d2edf14608c5c6b0270047512216"}
Jul 28 15:27:07.978941 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:07.978754    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79335509b0aceb1d41cd694799a499e1da84d2edf14608c5c6b0270047512216"
Jul 28 15:27:12.467522 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.467489    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"]
Jul 28 15:27:12.467901 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.467721    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x" podUID="4a27e5f1-3401-403a-9ca1-0d30b6c4c743" containerName="payload-processing" containerID="cri-o://a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e" gracePeriod=30
Jul 28 15:27:12.710232 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.710199    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:27:12.781604 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.781526    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-plugins-config-volume\") pod \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") "
Jul 28 15:27:12.781604 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.781571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mprcr\" (UniqueName: \"kubernetes.io/projected/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-kube-api-access-mprcr\") pod \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\" (UID: \"4a27e5f1-3401-403a-9ca1-0d30b6c4c743\") "
Jul 28 15:27:12.781880 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.781859    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a27e5f1-3401-403a-9ca1-0d30b6c4c743" (UID: "4a27e5f1-3401-403a-9ca1-0d30b6c4c743"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:12.783619 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.783596    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-kube-api-access-mprcr" (OuterVolumeSpecName: "kube-api-access-mprcr") pod "4a27e5f1-3401-403a-9ca1-0d30b6c4c743" (UID: "4a27e5f1-3401-403a-9ca1-0d30b6c4c743"). InnerVolumeSpecName "kube-api-access-mprcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 15:27:12.863839 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.863724    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"]
Jul 28 15:27:12.865693 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.865673    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-034645-ch6gc"]
Jul 28 15:27:12.882507 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.882484    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-plugins-config-volume\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:27:12.882507 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:12.882505    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mprcr\" (UniqueName: \"kubernetes.io/projected/4a27e5f1-3401-403a-9ca1-0d30b6c4c743-kube-api-access-mprcr\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:27:13.004458 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.004422    2579 generic.go:356] "Generic (PLEG): container finished" podID="4a27e5f1-3401-403a-9ca1-0d30b6c4c743" containerID="a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e" exitCode=0
Jul 28 15:27:13.004611 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.004463    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"
Jul 28 15:27:13.004611 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.004508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x" event={"ID":"4a27e5f1-3401-403a-9ca1-0d30b6c4c743","Type":"ContainerDied","Data":"a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e"}
Jul 28 15:27:13.004611 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.004553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x" event={"ID":"4a27e5f1-3401-403a-9ca1-0d30b6c4c743","Type":"ContainerDied","Data":"219a97994b9928f94c85e7d1f4657e1d10a3c1e5b4177b761ea791ae9c03b57e"}
Jul 28 15:27:13.004611 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.004577    2579 scope.go:117] "RemoveContainer" containerID="a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e"
Jul 28 15:27:13.013366 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.013346    2579 scope.go:117] "RemoveContainer" containerID="a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e"
Jul 28 15:27:13.013629 ip-10-0-141-50 kubenswrapper[2579]: E0728 15:27:13.013611    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e\": container with ID starting with a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e not found: ID does not exist" containerID="a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e"
Jul 28 15:27:13.013693 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.013638    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e"} err="failed to get container status \"a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e\": rpc error: code = NotFound desc = could not find container \"a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e\": container with ID starting with a6e002beab4dcd1a0907835b41078a19beaec02c46ebfa0c60256c1f7fee099e not found: ID does not exist"
Jul 28 15:27:13.027656 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.027631    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"]
Jul 28 15:27:13.031817 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:13.031767    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-034645-5fd4d46b96-fwl8x"]
Jul 28 15:27:14.192754 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:14.192722    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a27e5f1-3401-403a-9ca1-0d30b6c4c743" path="/var/lib/kubelet/pods/4a27e5f1-3401-403a-9ca1-0d30b6c4c743/volumes"
Jul 28 15:27:14.193125 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:14.193045    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54ad253b-6525-421f-b9fb-004a34a1bd01" path="/var/lib/kubelet/pods/54ad253b-6525-421f-b9fb-004a34a1bd01/volumes"
Jul 28 15:27:20.964577 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.964538    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"]
Jul 28 15:27:20.964967 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.964914    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a27e5f1-3401-403a-9ca1-0d30b6c4c743" containerName="payload-processing"
Jul 28 15:27:20.964967 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.964926    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a27e5f1-3401-403a-9ca1-0d30b6c4c743" containerName="payload-processing"
Jul 28 15:27:20.964967 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.964951    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54ad253b-6525-421f-b9fb-004a34a1bd01" containerName="revoke-keys"
Jul 28 15:27:20.964967 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.964957    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="54ad253b-6525-421f-b9fb-004a34a1bd01" containerName="revoke-keys"
Jul 28 15:27:20.965103 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.965008    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="54ad253b-6525-421f-b9fb-004a34a1bd01" containerName="revoke-keys"
Jul 28 15:27:20.965103 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.965015    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a27e5f1-3401-403a-9ca1-0d30b6c4c743" containerName="payload-processing"
Jul 28 15:27:20.969378 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.969363    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"
Jul 28 15:27:20.972601 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.972581    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cbpmp\""
Jul 28 15:27:20.972726 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.972611    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 15:27:20.977130 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:20.977107    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"]
Jul 28 15:27:21.060284 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:21.060256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27fd3132-56bd-4971-91c9-4d13936f33b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp\" (UID: \"27fd3132-56bd-4971-91c9-4d13936f33b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"
Jul 28 15:27:21.160846 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:21.160806    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27fd3132-56bd-4971-91c9-4d13936f33b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp\" (UID: \"27fd3132-56bd-4971-91c9-4d13936f33b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"
Jul 28 15:27:21.161512 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:21.161489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27fd3132-56bd-4971-91c9-4d13936f33b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp\" (UID: \"27fd3132-56bd-4971-91c9-4d13936f33b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"
Jul 28 15:27:21.281345 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:21.281244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"
Jul 28 15:27:21.435579 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:21.435541    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"]
Jul 28 15:27:21.445871 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:27:21.445847    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fd3132_56bd_4971_91c9_4d13936f33b6.slice/crio-72f8db70b81d377a40c8a7d068323deeaa3f0d4164da1e91cd0d808879698dc8 WatchSource:0}: Error finding container 72f8db70b81d377a40c8a7d068323deeaa3f0d4164da1e91cd0d808879698dc8: Status 404 returned error can't find the container with id 72f8db70b81d377a40c8a7d068323deeaa3f0d4164da1e91cd0d808879698dc8
Jul 28 15:27:22.052948 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:22.052912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp" event={"ID":"27fd3132-56bd-4971-91c9-4d13936f33b6","Type":"ContainerStarted","Data":"579b2006f9a4673ce007d003221339230f967ca29025a88fc0003138694ddaa5"}
Jul 28 15:27:22.053386 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:22.052976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp" event={"ID":"27fd3132-56bd-4971-91c9-4d13936f33b6","Type":"ContainerStarted","Data":"72f8db70b81d377a40c8a7d068323deeaa3f0d4164da1e91cd0d808879698dc8"}
Jul 28 15:27:22.071264 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:22.071201    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp" podStartSLOduration=2.071187559 podStartE2EDuration="2.071187559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:27:22.069839446 +0000 UTC m=+4112.357700401" watchObservedRunningTime="2026-07-28 15:27:22.071187559 +0000 UTC m=+4112.359048504"
Jul 28 15:27:23.059725 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:23.059692    2579 generic.go:356] "Generic (PLEG): container finished" podID="27fd3132-56bd-4971-91c9-4d13936f33b6" containerID="579b2006f9a4673ce007d003221339230f967ca29025a88fc0003138694ddaa5" exitCode=0
Jul 28 15:27:23.060112 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:23.059778    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp" event={"ID":"27fd3132-56bd-4971-91c9-4d13936f33b6","Type":"ContainerDied","Data":"579b2006f9a4673ce007d003221339230f967ca29025a88fc0003138694ddaa5"}
Jul 28 15:27:24.192238 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:24.192190    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"
Jul 28 15:27:24.292872 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:24.292837    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27fd3132-56bd-4971-91c9-4d13936f33b6-maas-api-service-ca\") pod \"27fd3132-56bd-4971-91c9-4d13936f33b6\" (UID: \"27fd3132-56bd-4971-91c9-4d13936f33b6\") "
Jul 28 15:27:24.293232 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:24.293190    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27fd3132-56bd-4971-91c9-4d13936f33b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27fd3132-56bd-4971-91c9-4d13936f33b6" (UID: "27fd3132-56bd-4971-91c9-4d13936f33b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 15:27:24.393932 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:24.393850    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27fd3132-56bd-4971-91c9-4d13936f33b6-maas-api-service-ca\") on node \"ip-10-0-141-50.ec2.internal\" DevicePath \"\""
Jul 28 15:27:25.070297 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:25.070268    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"
Jul 28 15:27:25.070297 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:25.070280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp" event={"ID":"27fd3132-56bd-4971-91c9-4d13936f33b6","Type":"ContainerDied","Data":"72f8db70b81d377a40c8a7d068323deeaa3f0d4164da1e91cd0d808879698dc8"}
Jul 28 15:27:25.070518 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:25.070334    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72f8db70b81d377a40c8a7d068323deeaa3f0d4164da1e91cd0d808879698dc8"
Jul 28 15:27:29.167563 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:29.167532    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"]
Jul 28 15:27:29.169080 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:29.169060    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d96a19-97bhp"]
Jul 28 15:27:30.192578 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:30.192541    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27fd3132-56bd-4971-91c9-4d13936f33b6" path="/var/lib/kubelet/pods/27fd3132-56bd-4971-91c9-4d13936f33b6/volumes"
Jul 28 15:27:53.810659 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:27:53.810632    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6988c6fd4-5qhnh_8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3/authorino/0.log"
Jul 28 15:28:00.303151 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:00.303121    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-lkbmk_f59a6f01-2e62-4648-97e7-db5cd5047a2b/copy-manifests/0.log"
Jul 28 15:28:00.311928 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:00.311908    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-lkbmk_f59a6f01-2e62-4648-97e7-db5cd5047a2b/manager/0.log"
Jul 28 15:28:02.643753 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:02.643721    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6988c6fd4-5qhnh_8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3/authorino/0.log"
Jul 28 15:28:03.275660 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:03.275629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wkpq4_2ff9a922-cd91-4600-988c-276b1a848b0e/manager/0.log"
Jul 28 15:28:03.531690 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:03.531603    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5j4fh_a91dbff4-a01e-4086-acb6-d95e1180837c/manager/0.log"
Jul 28 15:28:04.146782 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:04.146744    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-25fcb_e2d81f2b-8cd7-466d-9b4e-ec7894c2c659/kube-auth-proxy/0.log"
Jul 28 15:28:05.380029 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:05.380000    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-swbxm_8d46779a-f1cd-44fd-9b07-129276142eae/main/0.log"
Jul 28 15:28:05.748049 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:05.748020    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-29k2r_42ae94c2-5ee5-4967-acb9-ac73aca7402d/main/0.log"
Jul 28 15:28:14.704890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:14.704856    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d6pqw_2f352d50-ad34-4f97-b7d6-3e3d9cdf31d6/global-pull-secret-syncer/0.log"
Jul 28 15:28:14.982635 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:14.982563    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fz56r_857be85b-087b-4747-8d62-1a88529bba02/konnectivity-agent/0.log"
Jul 28 15:28:15.189332 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:15.189303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-50.ec2.internal_2efc845f7ae190bc5a73ce1b56253af6/haproxy/0.log"
Jul 28 15:28:21.117728 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:21.117646    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6988c6fd4-5qhnh_8a9f8324-1c9e-43a5-ab2d-ec2b3cf3d6e3/authorino/0.log"
Jul 28 15:28:22.004134 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:22.004105    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wkpq4_2ff9a922-cd91-4600-988c-276b1a848b0e/manager/0.log"
Jul 28 15:28:22.247910 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:22.247882    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5j4fh_a91dbff4-a01e-4086-acb6-d95e1180837c/manager/0.log"
Jul 28 15:28:23.453628 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.453590    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1192e4e6-7d06-45d0-b703-69d87e0dce78/alertmanager/0.log"
Jul 28 15:28:23.527297 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.527266    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1192e4e6-7d06-45d0-b703-69d87e0dce78/config-reloader/0.log"
Jul 28 15:28:23.568989 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.568953    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"]
Jul 28 15:28:23.569428 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.569411    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27fd3132-56bd-4971-91c9-4d13936f33b6" containerName="revoke-keys"
Jul 28 15:28:23.569528 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.569430    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="27fd3132-56bd-4971-91c9-4d13936f33b6" containerName="revoke-keys"
Jul 28 15:28:23.569584 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.569545    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="27fd3132-56bd-4971-91c9-4d13936f33b6" containerName="revoke-keys"
Jul 28 15:28:23.572327 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.572308    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.576734 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.576717    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hzqjc\"/\"openshift-service-ca.crt\""
Jul 28 15:28:23.576843 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.576751    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hzqjc\"/\"default-dockercfg-vtfl4\""
Jul 28 15:28:23.578018 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.578001    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hzqjc\"/\"kube-root-ca.crt\""
Jul 28 15:28:23.589327 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.589308    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"]
Jul 28 15:28:23.637958 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.637937    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1192e4e6-7d06-45d0-b703-69d87e0dce78/kube-rbac-proxy-web/0.log"
Jul 28 15:28:23.684101 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.684077    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1192e4e6-7d06-45d0-b703-69d87e0dce78/kube-rbac-proxy/0.log"
Jul 28 15:28:23.726839 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.726770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-proc\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.726839 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.726815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-lib-modules\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.726994 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.726872    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-sys\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.726994 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.726927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-podres\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.726994 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.726945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbk4t\" (UniqueName: \"kubernetes.io/projected/507795b2-e684-4506-9d62-2d6fabeec7b5-kube-api-access-kbk4t\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.746751 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.746728    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1192e4e6-7d06-45d0-b703-69d87e0dce78/kube-rbac-proxy-metric/0.log"
Jul 28 15:28:23.787941 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.787917    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1192e4e6-7d06-45d0-b703-69d87e0dce78/prom-label-proxy/0.log"
Jul 28 15:28:23.825466 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.825444    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1192e4e6-7d06-45d0-b703-69d87e0dce78/init-config-reloader/0.log"
Jul 28 15:28:23.827764 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-lib-modules\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.827867 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-sys\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.827867 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-podres\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.827867 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbk4t\" (UniqueName: \"kubernetes.io/projected/507795b2-e684-4506-9d62-2d6fabeec7b5-kube-api-access-kbk4t\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.828025 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827903    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-lib-modules\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.828025 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827923    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-sys\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.828025 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-proc\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.828025 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.827965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-podres\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.828025 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.828004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/507795b2-e684-4506-9d62-2d6fabeec7b5-proc\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.838978 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.838959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbk4t\" (UniqueName: \"kubernetes.io/projected/507795b2-e684-4506-9d62-2d6fabeec7b5-kube-api-access-kbk4t\") pod \"perf-node-gather-daemonset-pnxww\" (UID: \"507795b2-e684-4506-9d62-2d6fabeec7b5\") " pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:23.882046 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:23.882024    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:24.033463 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.033433    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"]
Jul 28 15:28:24.034288 ip-10-0-141-50 kubenswrapper[2579]: W0728 15:28:24.034265    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod507795b2_e684_4506_9d62_2d6fabeec7b5.slice/crio-1395e59c3e30f39f153b3e230e350a16654e0896759ed6e32d8fb567aa20354d WatchSource:0}: Error finding container 1395e59c3e30f39f153b3e230e350a16654e0896759ed6e32d8fb567aa20354d: Status 404 returned error can't find the container with id 1395e59c3e30f39f153b3e230e350a16654e0896759ed6e32d8fb567aa20354d
Jul 28 15:28:24.176945 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.176918    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5559d6958b-bp8cq_09d54cc9-7641-4acd-b3e0-d33dfb12d2c6/metrics-server/0.log"
Jul 28 15:28:24.367756 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.367671    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww" event={"ID":"507795b2-e684-4506-9d62-2d6fabeec7b5","Type":"ContainerStarted","Data":"1eff9f487c3b10514eb9ac4063d873f83627e0b2d826bb606c3e0d02393c3aee"}
Jul 28 15:28:24.367756 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.367721    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww" event={"ID":"507795b2-e684-4506-9d62-2d6fabeec7b5","Type":"ContainerStarted","Data":"1395e59c3e30f39f153b3e230e350a16654e0896759ed6e32d8fb567aa20354d"}
Jul 28 15:28:24.367756 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.367751    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:24.407370 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.407344    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hbt98_ce878207-39d8-47f3-9f53-fc06aa79bc68/node-exporter/0.log"
Jul 28 15:28:24.448514 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.448490    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hbt98_ce878207-39d8-47f3-9f53-fc06aa79bc68/kube-rbac-proxy/0.log"
Jul 28 15:28:24.485823 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.485799    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hbt98_ce878207-39d8-47f3-9f53-fc06aa79bc68/init-textfile/0.log"
Jul 28 15:28:24.661176 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.661082    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-8vfnl_9f755e95-904a-44c7-9409-2d05918e0360/kube-rbac-proxy-main/0.log"
Jul 28 15:28:24.696743 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.696713    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-8vfnl_9f755e95-904a-44c7-9409-2d05918e0360/kube-rbac-proxy-self/0.log"
Jul 28 15:28:24.736896 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.736870    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-8vfnl_9f755e95-904a-44c7-9409-2d05918e0360/openshift-state-metrics/0.log"
Jul 28 15:28:24.784457 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.784428    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f/prometheus/0.log"
Jul 28 15:28:24.822519 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.822490    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f/config-reloader/0.log"
Jul 28 15:28:24.858131 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.858097    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f/thanos-sidecar/0.log"
Jul 28 15:28:24.896334 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.896306    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f/kube-rbac-proxy-web/0.log"
Jul 28 15:28:24.928465 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.928391    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f/kube-rbac-proxy/0.log"
Jul 28 15:28:24.956925 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.956895    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f/kube-rbac-proxy-thanos/0.log"
Jul 28 15:28:24.986458 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:24.986436    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c91fa7f-4aeb-4fb3-bf06-1d3ff59b412f/init-config-reloader/0.log"
Jul 28 15:28:25.145554 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.145526    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-674d9c4964-c68d7_265fe75e-edbd-414d-81cd-c08949d3627c/telemeter-client/0.log"
Jul 28 15:28:25.177805 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.177779    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-674d9c4964-c68d7_265fe75e-edbd-414d-81cd-c08949d3627c/reload/0.log"
Jul 28 15:28:25.223779 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.223757    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-674d9c4964-c68d7_265fe75e-edbd-414d-81cd-c08949d3627c/kube-rbac-proxy/0.log"
Jul 28 15:28:25.275139 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.275115    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bb8897c9b-bd2h7_be9e4d99-4053-4482-b180-598af0463f71/thanos-query/0.log"
Jul 28 15:28:25.327731 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.327701    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bb8897c9b-bd2h7_be9e4d99-4053-4482-b180-598af0463f71/kube-rbac-proxy-web/0.log"
Jul 28 15:28:25.381705 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.381682    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bb8897c9b-bd2h7_be9e4d99-4053-4482-b180-598af0463f71/kube-rbac-proxy/0.log"
Jul 28 15:28:25.424198 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.424172    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bb8897c9b-bd2h7_be9e4d99-4053-4482-b180-598af0463f71/prom-label-proxy/0.log"
Jul 28 15:28:25.470256 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.470236    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bb8897c9b-bd2h7_be9e4d99-4053-4482-b180-598af0463f71/kube-rbac-proxy-rules/0.log"
Jul 28 15:28:25.519843 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:25.519785    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bb8897c9b-bd2h7_be9e4d99-4053-4482-b180-598af0463f71/kube-rbac-proxy-metrics/0.log"
Jul 28 15:28:27.124030 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:27.124002    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/1.log"
Jul 28 15:28:27.131002 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:27.130981    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-gl246_3513d484-71fc-4da3-8021-4260e01b51b1/console-operator/2.log"
Jul 28 15:28:27.688913 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:27.688886    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-brnr7_beb56aef-8622-474e-85b4-7a7d265fca68/download-server/0.log"
Jul 28 15:28:29.529147 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:29.529118    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hxz4r_9bcedd50-8330-4d98-98c7-1df1500b80bb/dns/0.log"
Jul 28 15:28:29.555224 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:29.555189    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hxz4r_9bcedd50-8330-4d98-98c7-1df1500b80bb/kube-rbac-proxy/0.log"
Jul 28 15:28:29.651618 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:29.651592    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qc8v5_70597690-c107-43a7-aff6-ac3afac91223/dns-node-resolver/0.log"
Jul 28 15:28:30.381885 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:30.381858    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww"
Jul 28 15:28:30.388286 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:30.388263    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wnx4d_3e2c8928-9386-42c8-9769-f3eaf4d71d78/node-ca/0.log"
Jul 28 15:28:30.422524 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:30.422481    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hzqjc/perf-node-gather-daemonset-pnxww" podStartSLOduration=7.422466284 podStartE2EDuration="7.422466284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 15:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:28:24.412937267 +0000 UTC m=+4174.700798211" watchObservedRunningTime="2026-07-28 15:28:30.422466284 +0000 UTC m=+4180.710327228"
Jul 28 15:28:31.593743 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:31.593710    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-25fcb_e2d81f2b-8cd7-466d-9b4e-ec7894c2c659/kube-auth-proxy/0.log"
Jul 28 15:28:32.843418 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:32.843388    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-br86x_d77ee87e-9bae-4382-bee7-14f28427ff6b/serve-healthcheck-canary/0.log"
Jul 28 15:28:33.632420 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:33.632393    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8sngc_307891dc-8150-4d47-8fa3-849b4b25c3dc/kube-rbac-proxy/0.log"
Jul 28 15:28:33.662790 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:33.662758    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8sngc_307891dc-8150-4d47-8fa3-849b4b25c3dc/exporter/0.log"
Jul 28 15:28:33.688813 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:33.688794    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8sngc_307891dc-8150-4d47-8fa3-849b4b25c3dc/extractor/0.log"
Jul 28 15:28:34.448030 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:34.448006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-25fcb_e2d81f2b-8cd7-466d-9b4e-ec7894c2c659/kube-auth-proxy/0.log"
Jul 28 15:28:36.133603 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:36.133572    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c4fff746-25fcb_e2d81f2b-8cd7-466d-9b4e-ec7894c2c659/kube-auth-proxy/0.log"
Jul 28 15:28:37.357890 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:37.357861    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-lkbmk_f59a6f01-2e62-4648-97e7-db5cd5047a2b/manager/0.log"
Jul 28 15:28:37.388692 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:37.388668    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f7956c87-lkbmk_f59a6f01-2e62-4648-97e7-db5cd5047a2b/copy-manifests/0.log"
Jul 28 15:28:38.640308 ip-10-0-141-50 kubenswrapper[2579]: I0728 15:28:38.640278    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5d8c94b858-xcfs8_e8c2ef7f-5de6-44c4-bdb3-090998c86d72/manager/0.log"