Jul 24 23:29:24.357229 ip-10-0-141-134 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 23:29:25.744963 ip-10-0-141-134 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 23:29:25.744963 ip-10-0-141-134 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 23:29:25.744963 ip-10-0-141-134 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 23:29:25.744963 ip-10-0-141-134 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 23:29:25.744963 ip-10-0-141-134 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 23:29:25.746644 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.746546    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 23:29:25.752189 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752157    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:25.752189 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752184    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:25.752189 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752188    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:25.752189 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752193    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:25.752189 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752196    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:25.752189 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752200    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752204    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752209    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752212    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752215    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752218    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752221    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752225    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752231    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752234    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752237    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752239    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752242    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752245    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752248    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752250    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752253    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752256    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752259    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752262    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:25.752429 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752264    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752269    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752272    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752275    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752277    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752280    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752283    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752286    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752288    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752291    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752294    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752297    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752299    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752305    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752309    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752313    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752317    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752320    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752322    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:25.752919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752325    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752329    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752333    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752336    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752341    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752344    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752350    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752353    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752356    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752359    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752361    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752364    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752367    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752370    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752373    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752376    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752380    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752384    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752389    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:25.753394 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752392    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752395    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752398    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752401    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752404    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752407    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752409    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752412    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752416    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752419    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752422    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752425    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752431    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752433    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752436    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752439    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752442    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752445    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752448    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:25.753868 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752466    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752469    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752471    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752475    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752480    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752482    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752485    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752488    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752491    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752495    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752499    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752501    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752504    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752507    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752510    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752513    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.752516    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753687    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753698    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753702    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:25.754329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753705    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753708    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753711    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753714    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753719    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753722    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753725    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753728    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753730    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753734    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753736    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753742    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753745    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753747    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753750    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753753    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753755    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753758    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753760    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:25.754843 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753765    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753769    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753772    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753775    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753780    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753783    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753786    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753789    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753791    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753794    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753797    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753799    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753802    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753805    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753809    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753812    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753815    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753820    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753824    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:25.755330 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753826    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753829    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753831    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753834    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753837    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753839    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753842    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753845    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753849    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753852    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753857    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753861    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753864    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753868    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753871    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753874    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753877    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753880    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753883    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753886    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:25.755824 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753888    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753891    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753898    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753903    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753906    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753908    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753911    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753915    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753918    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753920    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753923    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753925    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753929    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753934    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753937    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753939    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753942    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753944    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753947    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753950    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:25.756326 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753952    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753955    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753958    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753960    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753963    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753966    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753971    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753974    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753976    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753979    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753982    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753984    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753987    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753990    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753992    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753995    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.753997    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754000    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754655    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:25.756853 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754661    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754666    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754673    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754676    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754680    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754683    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754686    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754689    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754693    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754695    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754698    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754701    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754704    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754707    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754712    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754715    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754717    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754721    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754724    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:25.757333 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754727    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754730    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754733    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754736    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754738    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754741    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754744    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754746    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754752    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754754    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754757    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754761    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754764    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754767    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754770    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754772    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754775    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754779    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754781    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754787    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754790    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:25.757849 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754793    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754795    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754798    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754801    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754803    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754806    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754808    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754811    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754813    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754816    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754819    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754824    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754827    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754829    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754832    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754834    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754837    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754840    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754842    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:25.758349 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754845    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754849    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754853    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754858    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754864    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754867    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754869    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754872    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754875    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754878    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754880    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754883    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754886    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754889    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754891    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754894    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754896    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754901    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754904    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:25.758835 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754907    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754916    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754919    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754922    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754924    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754927    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754930    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754932    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754935    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754940    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754943    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754945    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754948    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754951    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754954    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754956    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754960    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754963    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754966    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.754969    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:25.759329 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755182    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755203    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755212    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755354    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755360    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755364    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755369    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755374    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755377    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755382    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755386    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755389    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755392    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755395    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755399    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755402    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755405    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755409    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755414    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755417    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755421    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755424    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755428    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755432    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 23:29:25.759831 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755435    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755438    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755442    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755445    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755449    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755463    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755466    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755470    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755475    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755479    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755482    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755485    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755489    2572 flags.go:64] FLAG: --enable-server="true"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755492    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755498    2572 flags.go:64] FLAG: --event-burst="100"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755501    2572 flags.go:64] FLAG: --event-qps="50"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755505    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755508    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755511    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755517    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755522    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755525    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755530    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755534    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755538    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 23:29:25.760466 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755543    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755547    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755550    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755555    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755560    2572 flags.go:64] FLAG: --feature-gates=""
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755564    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755570    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755575    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755578    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755581    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755584    2572 flags.go:64] FLAG: --help="false"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755588    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-141-134.ec2.internal"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755591    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755594    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755597    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755601    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755606    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755610    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755613    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755616    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755619    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755623    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755626    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755629    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 24 23:29:25.761142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755632    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755636    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755639    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755641    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755644    2572 flags.go:64] FLAG: --lock-file=""
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755647    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755650    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755654    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755664    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755669    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755673    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755677    2572 flags.go:64] FLAG: --logging-format="text"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755680    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755684    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755687    2572 flags.go:64] FLAG: --manifest-url=""
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755690    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755696    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755699    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755703    2572 flags.go:64] FLAG: --max-pods="110"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755706    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755710    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755713    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755716    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755719    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755722    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 23:29:25.761743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755725    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755733    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755737    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755740    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755743    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755746    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755752    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755755    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755758    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755761    2572 flags.go:64] FLAG: --port="10250"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755764    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755767    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07d827d786230a1c8"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755770    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755773    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755776    2572 flags.go:64] FLAG: --register-node="true"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755779    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755783    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755787    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755790    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755793    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755796    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755800    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755803    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755806    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755809    2572 flags.go:64] FLAG: --runonce="false"
Jul 24 23:29:25.762442 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755812    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755816    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755819    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755822    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755825    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755829    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755836    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755840    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755843    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755847    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755850    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755853    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755856    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755859    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755862    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755869    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755872    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755881    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755887    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755891    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755894    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755897    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755900    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755903    2572 flags.go:64] FLAG: --v="2"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755907    2572 flags.go:64] FLAG: --version="false"
Jul 24 23:29:25.763183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755912    2572 flags.go:64] FLAG: --vmodule=""
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755916    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.755920    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756052    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756059    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756062    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756068    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756070    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756073    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756077    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756080    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756083    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756086    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756091    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756094    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756097    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756099    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756103    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756107    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756111    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:25.763836 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756114    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756117    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756120    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756122    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756126    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756128    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756131    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756134    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756137    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756139    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756144    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756148    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756151    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756154    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756157    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756160    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756163    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756166    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756170    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:25.764389 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756174    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756176    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756179    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756182    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756185    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756188    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756192    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756194    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756197    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756200    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756203    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756205    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756208    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756211    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756213    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756216    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756219    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756221    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756224    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756226    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:25.764870 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756229    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756232    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756235    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756237    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756240    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756243    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756245    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756255    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756258    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756261    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756264    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756267    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756270    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756273    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756276    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756278    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756281    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756284    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756288    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756291    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:25.765374 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756293    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756296    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756299    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756302    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756304    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756307    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756309    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756312    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756315    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756317    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756320    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756323    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756326    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756329    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756331    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756334    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756337    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756339    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756342    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756345    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:25.765877 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756349    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756351    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.756355    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.757013    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.764485    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.764506    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.765247    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.765256    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765319    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765327    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765331    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765334    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765337    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765340    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:25.766365 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765343    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765345    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765349    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765351    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765354    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765357    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765359    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765362    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765365    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765368    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765370    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765373    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765376    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765378    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765382    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765385    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765388    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765391    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765393    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:25.766798 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765397    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765401    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765404    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765408    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765412    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765415    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765417    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765420    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765422    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765425    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765428    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765430    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765433    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765436    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765438    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765441    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765445    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765447    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765464    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765468    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:25.767272 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765472    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765476    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765480    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765483    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765486    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765489    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765492    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765494    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765497    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765500    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765502    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765505    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765508    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765510    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765513    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765516    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765519    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765523    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765526    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765529    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765531    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:25.767779 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765534    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765537    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765540    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765542    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765545    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765547    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765550    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765552    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765555    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765557    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765560    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765563    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765565    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765568    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765570    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765574    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765578    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765581    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765584    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:25.768329 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765587    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765590    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765593    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765596    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765599    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765601    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765604    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765607    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765610    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765613    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765616    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765619    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765622    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765624    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.765630    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 23:29:25.768879 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765781    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765788    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765792    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765795    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765799    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765802    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765804    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765807    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765810    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765813    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765816    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765819    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765822    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765824    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765829    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765832    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765835    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765838    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765841    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:25.769264 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765844    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765846    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765849    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765851    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765854    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765857    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765860    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765862    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765865    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765869    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765872    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765875    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765877    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765880    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765883    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765886    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765889    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765891    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765894    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765897    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:25.769763 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765899    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765902    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765904    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765907    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765910    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765912    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765915    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765918    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765921    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765923    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765926    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765929    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765931    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765934    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765936    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765939    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765942    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765944    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765947    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765949    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:25.770283 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765952    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765954    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765957    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765960    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765963    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765965    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765968    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765971    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765973    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765975    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765978    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765980    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765983    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765985    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765988    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765990    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765993    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765995    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.765998    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766001    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:25.770859 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766003    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766006    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766008    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766011    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766013    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766016    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766018    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766021    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766025    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766029    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766032    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766035    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766037    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766040    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766046    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766050    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766053    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766055    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766059    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:25.771356 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:25.766062    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:25.771845 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.766067    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 23:29:25.771845 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.766200    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 23:29:25.771845 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.768531    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 23:29:25.771845 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.769866    2572 server.go:1019] "Starting client certificate rotation"
Jul 24 23:29:25.771845 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.769974    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 23:29:25.771845 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.770521    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 23:29:25.772248 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.772228    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 23:29:25.773730 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.773709    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 23:29:25.782552 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.782529    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 24 23:29:25.787058 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.787042    2572 log.go:25] "Validated CRI v1 image API"
Jul 24 23:29:25.788643 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.788622    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 23:29:25.792065 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.792016    2572 fs.go:135] Filesystem UUIDs: map[3bab6551-0de3-41b0-a4f0-178ef698ad3b:/dev/nvme0n1p4 6deeec65-4367-4927-83bf-f3c971d0f54a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 24 23:29:25.792174 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.792062    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 23:29:25.798669 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.798537    2572 manager.go:216] Machine: {Timestamp:2026-07-24 23:29:25.796688276 +0000 UTC m=+0.292400142 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099721 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec261c06a682f85178b77e42bdbf84fe SystemUUID:ec261c06-a682-f851-78b7-7e42bdbf84fe BootID:6e8a7662-1b7c-407e-b7c5-e143b6ec4a9e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:ec:e8:bf:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:ec:e8:bf:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:65:d0:aa:45:db Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 23:29:25.798669 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.798652    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 23:29:25.798838 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.798760    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 23:29:25.800012 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.799995    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 23:29:25.801648 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.801612    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 23:29:25.801795 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.801653    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 23:29:25.802239 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.802230    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 23:29:25.802268 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.802242    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 23:29:25.802268 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.802249    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 23:29:25.802734 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.802723    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 23:29:25.804326 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.804313    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 23:29:25.808572 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.808554    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 23:29:25.808706 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.808697    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 23:29:25.808744 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.808730    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 23:29:25.808744 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.808740    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 23:29:25.808795 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.808755    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 23:29:25.808795 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.808765    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 24 23:29:25.808795 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.808775    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 23:29:25.809185 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.809172    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 23:29:25.809185 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.809192    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 23:29:25.811308 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.811269    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 23:29:25.813019 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.812957    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 23:29:25.813098 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.813032    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 23:29:25.814260 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814240    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 23:29:25.814260 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814260    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814267    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814273    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814282    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814288    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814296    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814301    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814308    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814314    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814322    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 23:29:25.814333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814328    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 23:29:25.814988 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814978    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 23:29:25.814988 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.814988    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 23:29:25.818376 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.818358    2572 server.go:1268] "Started kubelet"
Jul 24 23:29:25.818483 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.818391    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 23:29:25.818875 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.818527    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 23:29:25.818875 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.818579    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 23:29:25.818875 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.818602    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 23:29:25.818875 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.818737    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 23:29:25.818875 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.818803    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 23:29:25.819888 ip-10-0-141-134 systemd[1]: Started Kubernetes Kubelet.
Jul 24 23:29:25.821027 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.820941    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 23:29:25.821586 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.821568    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 23:29:25.822786 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.822767    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dshkk"
Jul 24 23:29:25.825318 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.825299    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 23:29:25.826592 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.825749    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-134.ec2.internal.18c55d9f3c7dcab5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-134.ec2.internal,UID:ip-10-0-141-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-134.ec2.internal,},FirstTimestamp:2026-07-24 23:29:25.818329781 +0000 UTC m=+0.314041646,LastTimestamp:2026-07-24 23:29:25.818329781 +0000 UTC m=+0.314041646,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-134.ec2.internal,}"
Jul 24 23:29:25.828705 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.828684    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 23:29:25.828889 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.828869    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 23:29:25.829127 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.829099    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:25.829441 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.829407    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 23:29:25.829441 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.829427    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 23:29:25.829678 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.829658    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 23:29:25.830181 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830167    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 23:29:25.830181 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830179    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 23:29:25.830313 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830258    2572 factory.go:55] Registering systemd factory
Jul 24 23:29:25.830313 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830275    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 24 23:29:25.830464 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830422    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dshkk"
Jul 24 23:29:25.830674 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830657    2572 factory.go:153] Registering CRI-O factory
Jul 24 23:29:25.830747 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830677    2572 factory.go:223] Registration of the crio container factory successfully
Jul 24 23:29:25.830747 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830743    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 23:29:25.830837 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830771    2572 factory.go:103] Registering Raw factory
Jul 24 23:29:25.830837 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.830787    2572 manager.go:1196] Started watching for new ooms in manager
Jul 24 23:29:25.831241 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.831207    2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 23:29:25.831318 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.831258    2572 manager.go:349] Starting recovery of all containers
Jul 24 23:29:25.842660 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.842631    2572 manager.go:354] Recovery completed
Jul 24 23:29:25.844535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.844485    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:25.848708 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.848689    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:25.850788 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.850768    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:25.850901 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.850807    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:25.850901 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.850823    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:25.851473 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.851446    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 23:29:25.851473 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.851472    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 23:29:25.851604 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.851446    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:25.851604 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.851492    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 23:29:25.852490 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.852473    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:25.852628 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.852501    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:25.852628 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.852512    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:25.853570 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.853557    2572 policy_none.go:49] "None policy: Start"
Jul 24 23:29:25.853627 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.853578    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 23:29:25.853627 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.853592    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 23:29:25.855592 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.855577    2572 policy_none.go:47] "Start"
Jul 24 23:29:25.901356 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.901175    2572 manager.go:337] "Starting Device Plugin manager"
Jul 24 23:29:25.901527 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.901394    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 23:29:25.901527 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.901405    2572 server.go:87] "Starting device plugin registration server"
Jul 24 23:29:25.901704 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.901690    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 23:29:25.901746 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.901705    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 23:29:25.901822 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.901807    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 23:29:25.901967 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.901898    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 23:29:25.901967 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.901913    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 23:29:25.902587 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.902565    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 23:29:25.902689 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.902626    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:25.958820 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.958786    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 23:29:25.961049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.960027    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 23:29:25.961049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.960073    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 23:29:25.961049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.960111    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 23:29:25.961049 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:25.960156    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 23:29:25.962447 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:25.962414    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:26.002438 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.002401    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:26.003801 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.003779    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:26.003970 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.003818    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:26.003970 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.003833    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:26.003970 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.003873    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.011305 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.011278    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.011305 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.011309    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-134.ec2.internal\": node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.025030 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.025002    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.061156 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.061118    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"]
Jul 24 23:29:26.061329 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.061209    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:26.062256 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.062239    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:26.062371 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.062275    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:26.062371 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.062288    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:26.064722 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.064703    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:26.064833 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.064815    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.064885 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.064854    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:26.065597 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.065583    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:26.065669 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.065609    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:26.065669 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.065620    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:26.066181 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.066166    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:26.066240 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.066195    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:26.066240 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.066208    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:26.068531 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.068516    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.068595 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.068547    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:26.069358 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.069335    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:26.069435 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.069364    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:26.069435 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.069386    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:26.092342 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.092316    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-134.ec2.internal\" not found" node="ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.097143 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.097124    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-134.ec2.internal\" not found" node="ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.125825 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.125791    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.132237 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.132176    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c14bfa1a5ff22178fc8fb6624f7a3b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"7c14bfa1a5ff22178fc8fb6624f7a3b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.132237 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.132208    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c14bfa1a5ff22178fc8fb6624f7a3b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"7c14bfa1a5ff22178fc8fb6624f7a3b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.132237 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.132229    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/690e7b1c9b542cf29dc8b6b0d7b228e9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-134.ec2.internal\" (UID: \"690e7b1c9b542cf29dc8b6b0d7b228e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.226286 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.226253    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.232627 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.232597    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c14bfa1a5ff22178fc8fb6624f7a3b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"7c14bfa1a5ff22178fc8fb6624f7a3b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.232694 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.232629    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c14bfa1a5ff22178fc8fb6624f7a3b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"7c14bfa1a5ff22178fc8fb6624f7a3b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.232694 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.232646    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/690e7b1c9b542cf29dc8b6b0d7b228e9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-134.ec2.internal\" (UID: \"690e7b1c9b542cf29dc8b6b0d7b228e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.232694 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.232687    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/690e7b1c9b542cf29dc8b6b0d7b228e9-config\") pod \"kube-apiserver-proxy-ip-10-0-141-134.ec2.internal\" (UID: \"690e7b1c9b542cf29dc8b6b0d7b228e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.232801 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.232714    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c14bfa1a5ff22178fc8fb6624f7a3b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"7c14bfa1a5ff22178fc8fb6624f7a3b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.232801 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.232723    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c14bfa1a5ff22178fc8fb6624f7a3b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"7c14bfa1a5ff22178fc8fb6624f7a3b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.327069 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.327035    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.395590 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.395521    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.401665 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.401355    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.428047 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.428012    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.528592 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.528547    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.629065 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.629031    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.729695 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.729612    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 24 23:29:26.743737 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.743713    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:26.769842 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.769805    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 23:29:26.770267 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.769959    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 23:29:26.770267 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.769961    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 23:29:26.770267 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.769963    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 23:29:26.789241 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.789207    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:26.809064 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.809030    2572 apiserver.go:52] "Watching apiserver"
Jul 24 23:29:26.814770 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.814746    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 23:29:26.815957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.815929    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49","openshift-dns/node-resolver-c427z","openshift-multus/multus-additional-cni-plugins-jdcv2","openshift-multus/multus-zlbng","openshift-network-operator/iptables-alerter-72z7r","openshift-ovn-kubernetes/ovnkube-node-7qjmg","kube-system/global-pull-secret-syncer-n6fg5","kube-system/konnectivity-agent-nlz49","openshift-cluster-node-tuning-operator/tuned-nwsbv","openshift-image-registry/node-ca-6fwbl","openshift-multus/network-metrics-daemon-wvlgk","openshift-network-diagnostics/network-check-target-cqzrz"]
Jul 24 23:29:26.818383 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.818361    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.820535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.820509    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 23:29:26.820729 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.820698    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7nt64\""
Jul 24 23:29:26.820810 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.820704    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.821012 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.820706    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 23:29:26.821012 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.820776    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 23:29:26.822657 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.822641    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 23:29:26.822842 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.822825    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 23:29:26.822987 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.822972    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-79lvf\""
Jul 24 23:29:26.823225 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.823211    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.825269 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.825243    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 23:29:26.825375 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.825242    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 23:29:26.825375 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.825314    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 23:29:26.825375 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.825342    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 23:29:26.825560 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.825358    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c6sfz\""
Jul 24 23:29:26.825560 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.825555    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.825629 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.825614    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 23:29:26.827601 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.827584    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6cc4z\""
Jul 24 23:29:26.827670 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.827608    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 23:29:26.828093 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.828076    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.829159 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.829144    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 23:29:26.829552 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.829538    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.829884 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.829869    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 23:29:26.830079 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.830061    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 23:29:26.830149 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.830068    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 23:29:26.832672 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.832467    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-49kj6\""
Jul 24 23:29:26.833696 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.832959    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.835894 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.835783    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 23:29:26.835985 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.835964    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 23:29:26.836042 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.835985    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 23:29:26.836042 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.835991    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-systemd\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836042 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836022    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-registration-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.836194 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836057    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-device-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.836194 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836080    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-netns\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836194 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836110    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-cnibin\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.836194 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836112    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x89zs\""
Jul 24 23:29:26.836194 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-cni-multus\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836195    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836204    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836194    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-ovn\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836251    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-sys-fs\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836259    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-hostroot\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836317    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovn-node-metrics-cert\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836340    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836333    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 23:24:25 +0000 UTC" deadline="2028-02-07 07:36:49.756716295 +0000 UTC"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836394    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836397    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13496h7m22.920322996s"
Jul 24 23:29:26.836468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836392    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-daemon-config\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.836466    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836492    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-etc-kubernetes\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836508    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsv2r\" (UniqueName: \"kubernetes.io/projected/1b33f0ce-edef-42b5-a87f-da504d0c0862-kube-api-access-hsv2r\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836525    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-var-lib-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836549    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-socket-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836572    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-etc-selinux\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836608    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-kubelet\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836638    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2gbk\" (UniqueName: \"kubernetes.io/projected/92e1a018-5f68-4349-9b9b-7b6c3d267c53-kube-api-access-s2gbk\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836662    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-kubelet\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836682    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-multus-certs\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836703    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836755    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-cni-bin\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836774    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92e1a018-5f68-4349-9b9b-7b6c3d267c53-host-slash\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836790    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovnkube-config\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836814    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-env-overrides\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.836957 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836829    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-log-socket\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836842    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovnkube-script-lib\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836911    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-cni-netd\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836928    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnlr\" (UniqueName: \"kubernetes.io/projected/c4cac730-5c82-4f69-a783-34f405316b30-kube-api-access-qfnlr\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836946    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-etc-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.836989    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb8ph\" (UniqueName: \"kubernetes.io/projected/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-kube-api-access-zb8ph\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837017    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/608f5875-7843-48a2-a100-6050a0fdac6c-hosts-file\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837044    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837082    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837105    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-cni-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837127    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-cnibin\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837150    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b33f0ce-edef-42b5-a87f-da504d0c0862-cni-binary-copy\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837171    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-node-log\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837206    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-system-cni-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.837634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837245    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-os-release\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-socket-dir-parent\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837313    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-conf-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837337    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/92e1a018-5f68-4349-9b9b-7b6c3d267c53-iptables-alerter-script\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837362    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-slash\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837377    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-run-netns\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837391    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-cni-bin\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837420    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw2kj\" (UniqueName: \"kubernetes.io/projected/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-kube-api-access-hw2kj\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837448    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/608f5875-7843-48a2-a100-6050a0fdac6c-tmp-dir\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837507    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-k8s-cni-cncf-io\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837534    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khb4d\" (UniqueName: \"kubernetes.io/projected/608f5875-7843-48a2-a100-6050a0fdac6c-kube-api-access-khb4d\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837557    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-system-cni-dir\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837581    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-os-release\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837602    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837634    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-systemd-units\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.838162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.837660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.838758 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.838635    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:26.840545 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.840527    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 23:29:26.840643 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.840547    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 23:29:26.840643 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.840528    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xxzm7\""
Jul 24 23:29:26.841181 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.841165    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.841439 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.841410    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 23:29:26.841517 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.841505    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 24 23:29:26.843333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.843314    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 23:29:26.843407 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.843387    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-96jlv\""
Jul 24 23:29:26.843497 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.843480    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 23:29:26.843960 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.843944    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:26.844338 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.844323    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 23:29:26.845789 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.845774    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 23:29:26.845894 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.845879    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 23:29:26.845936 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.845884    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qdqgr\""
Jul 24 23:29:26.846000 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.845986    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 23:29:26.846647 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.846633    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:26.846714 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.846697    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:26.849176 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.849158    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"]
Jul 24 23:29:26.849294 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.849244    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:26.849325 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:26.849288    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:26.851265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.851248    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 23:29:26.851337 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.851323    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"]
Jul 24 23:29:26.865623 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.865602    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-clbmt"
Jul 24 23:29:26.872899 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.872877    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-clbmt"
Jul 24 23:29:26.931319 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.931288    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 23:29:26.937843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.937804    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-sys-fs\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.937843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.937841    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-hostroot\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938037 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.937966    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-sys-fs\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.938037 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.937972    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-hostroot\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938037 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.937999    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-var-lib-kubelet\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.938186 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938082    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6634d566-76cf-4c96-9668-bc6ee975ed83-kubelet-config\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:26.938186 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938120    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovn-node-metrics-cert\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.938186 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938158    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938196    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-daemon-config\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938221    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-etc-kubernetes\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938244    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsv2r\" (UniqueName: \"kubernetes.io/projected/1b33f0ce-edef-42b5-a87f-da504d0c0862-kube-api-access-hsv2r\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938272    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysctl-d\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938296    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938322    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-var-lib-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938352    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-socket-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.938392 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-etc-selinux\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938401    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-kubelet\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938428    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-host\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938470    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f912844-4661-427c-aa0a-139a17222d34-konnectivity-ca\") pod \"konnectivity-agent-nlz49\" (UID: \"7f912844-4661-427c-aa0a-139a17222d34\") " pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938496    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgvhm\" (UniqueName: \"kubernetes.io/projected/24645aa9-1df4-4033-84e5-00059d19bf6d-kube-api-access-pgvhm\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938525    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2gbk\" (UniqueName: \"kubernetes.io/projected/92e1a018-5f68-4349-9b9b-7b6c3d267c53-kube-api-access-s2gbk\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938529    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938538    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-kubelet\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938550    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-kubelet\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938577    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-multus-certs\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938602    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6634d566-76cf-4c96-9668-bc6ee975ed83-dbus\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938629    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938659    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-socket-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938659    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-cni-bin\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938707    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysctl-conf\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938719    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-etc-selinux\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-245k9\" (UniqueName: \"kubernetes.io/projected/c3d9d2d2-b041-4df6-b863-40cd44584799-kube-api-access-245k9\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.938769 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938710    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-cni-bin\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938784    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-etc-kubernetes\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938791    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-multus-certs\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938802    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938803    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938823    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92e1a018-5f68-4349-9b9b-7b6c3d267c53-host-slash\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938766    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-daemon-config\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938852    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovnkube-config\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938881    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-env-overrides\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938908    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-kubelet\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938945    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92e1a018-5f68-4349-9b9b-7b6c3d267c53-host-slash\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938970    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b76e78ba-3363-4b80-9ac2-0ff25f65088f-serviceca\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.938995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f912844-4661-427c-aa0a-139a17222d34-agent-certs\") pod \"konnectivity-agent-nlz49\" (UID: \"7f912844-4661-427c-aa0a-139a17222d34\") " pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939069    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-var-lib-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939072    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-log-socket\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939202    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovnkube-script-lib\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939211    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-log-socket\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.939468 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939265    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-systemd\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939291    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939327    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939354    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939388    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-cni-netd\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939406    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939414    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-env-overrides\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939436    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnlr\" (UniqueName: \"kubernetes.io/projected/c4cac730-5c82-4f69-a783-34f405316b30-kube-api-access-qfnlr\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939489    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939491    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-modprobe-d\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-tuned\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939548    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovnkube-config\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939559    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-etc-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939570    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-cni-netd\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939591    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb8ph\" (UniqueName: \"kubernetes.io/projected/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-kube-api-access-zb8ph\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939605    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-etc-openvswitch\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939642    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/608f5875-7843-48a2-a100-6050a0fdac6c-hosts-file\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.940282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939671    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939727    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/608f5875-7843-48a2-a100-6050a0fdac6c-hosts-file\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939761    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovnkube-script-lib\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939770    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939798    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-cni-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939855    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-cnibin\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939917    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-cnibin\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.939949    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b33f0ce-edef-42b5-a87f-da504d0c0862-cni-binary-copy\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940044    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-cni-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940081    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-node-log\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940146    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-node-log\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-system-cni-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940228    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-system-cni-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940256    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-os-release\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940301    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940329    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-os-release\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940328    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4cac730-5c82-4f69-a783-34f405316b30-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940356    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-socket-dir-parent\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940383    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-conf-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940412    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysconfig\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940437    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-lib-modules\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940442    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-socket-dir-parent\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940446    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b33f0ce-edef-42b5-a87f-da504d0c0862-cni-binary-copy\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940479    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/92e1a018-5f68-4349-9b9b-7b6c3d267c53-iptables-alerter-script\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940531    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-multus-conf-dir\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940574    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-slash\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940611    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-run-netns\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-cni-bin\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940659    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-slash\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940663    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw2kj\" (UniqueName: \"kubernetes.io/projected/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-kube-api-access-hw2kj\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940702    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/608f5875-7843-48a2-a100-6050a0fdac6c-tmp-dir\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940711    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-run-netns\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940725    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-k8s-cni-cncf-io\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-cni-bin\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940750    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3d9d2d2-b041-4df6-b863-40cd44584799-tmp\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.941904 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940788    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khb4d\" (UniqueName: \"kubernetes.io/projected/608f5875-7843-48a2-a100-6050a0fdac6c-kube-api-access-khb4d\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940815    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-system-cni-dir\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940833    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-k8s-cni-cncf-io\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940847    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-os-release\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940872    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940880    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-system-cni-dir\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940896    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-systemd-units\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940928    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940952    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-kubernetes\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940980    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.940946    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-os-release\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941005    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-systemd\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941030    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941033    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-systemd-units\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941040    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/608f5875-7843-48a2-a100-6050a0fdac6c-tmp-dir\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941047    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/92e1a018-5f68-4349-9b9b-7b6c3d267c53-iptables-alerter-script\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941040    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941039    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-registration-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941083    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-systemd\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941099    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-registration-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941102    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-device-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941134    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-netns\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941138    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-device-dir\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941186    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-run-netns\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941159    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-run\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941234    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b76e78ba-3363-4b80-9ac2-0ff25f65088f-host\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941261    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-cnibin\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941285    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-cni-multus\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941309    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-sys\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941328    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b33f0ce-edef-42b5-a87f-da504d0c0862-host-var-lib-cni-multus\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941336    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcs2s\" (UniqueName: \"kubernetes.io/projected/b76e78ba-3363-4b80-9ac2-0ff25f65088f-kube-api-access-zcs2s\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941363    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-ovn\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941330    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4cac730-5c82-4f69-a783-34f405316b30-cnibin\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.941437    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-run-ovn\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.942924 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.942306    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-ovn-node-metrics-cert\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.947497 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.947473    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2gbk\" (UniqueName: \"kubernetes.io/projected/92e1a018-5f68-4349-9b9b-7b6c3d267c53-kube-api-access-s2gbk\") pod \"iptables-alerter-72z7r\" (UID: \"92e1a018-5f68-4349-9b9b-7b6c3d267c53\") " pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:26.947695 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.947645    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb8ph\" (UniqueName: \"kubernetes.io/projected/b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b-kube-api-access-zb8ph\") pod \"ovnkube-node-7qjmg\" (UID: \"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:26.947807 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.947787    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsv2r\" (UniqueName: \"kubernetes.io/projected/1b33f0ce-edef-42b5-a87f-da504d0c0862-kube-api-access-hsv2r\") pod \"multus-zlbng\" (UID: \"1b33f0ce-edef-42b5-a87f-da504d0c0862\") " pod="openshift-multus/multus-zlbng"
Jul 24 23:29:26.947857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.947790    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnlr\" (UniqueName: \"kubernetes.io/projected/c4cac730-5c82-4f69-a783-34f405316b30-kube-api-access-qfnlr\") pod \"multus-additional-cni-plugins-jdcv2\" (UID: \"c4cac730-5c82-4f69-a783-34f405316b30\") " pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:26.948496 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.948476    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw2kj\" (UniqueName: \"kubernetes.io/projected/bf983c8d-2cd0-4bfc-b91a-215765ffebcb-kube-api-access-hw2kj\") pod \"aws-ebs-csi-driver-node-tzp49\" (UID: \"bf983c8d-2cd0-4bfc-b91a-215765ffebcb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:26.949288 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:26.949267    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khb4d\" (UniqueName: \"kubernetes.io/projected/608f5875-7843-48a2-a100-6050a0fdac6c-kube-api-access-khb4d\") pod \"node-resolver-c427z\" (UID: \"608f5875-7843-48a2-a100-6050a0fdac6c\") " pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:27.002273 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.002230    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c14bfa1a5ff22178fc8fb6624f7a3b8.slice/crio-151ff07976d2018198d2f5913691a25458d55a41a45b9ce10d18457c0fbe4d9b WatchSource:0}: Error finding container 151ff07976d2018198d2f5913691a25458d55a41a45b9ce10d18457c0fbe4d9b: Status 404 returned error can't find the container with id 151ff07976d2018198d2f5913691a25458d55a41a45b9ce10d18457c0fbe4d9b
Jul 24 23:29:27.003053 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.003027    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690e7b1c9b542cf29dc8b6b0d7b228e9.slice/crio-24900b8c97fbbec928e0f136852aff8b0e83d5c0802b31f5f60c7078ef29938d WatchSource:0}: Error finding container 24900b8c97fbbec928e0f136852aff8b0e83d5c0802b31f5f60c7078ef29938d: Status 404 returned error can't find the container with id 24900b8c97fbbec928e0f136852aff8b0e83d5c0802b31f5f60c7078ef29938d
Jul 24 23:29:27.007143 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.007126    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:29:27.042288 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042256    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-sys\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.042288 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcs2s\" (UniqueName: \"kubernetes.io/projected/b76e78ba-3363-4b80-9ac2-0ff25f65088f-kube-api-access-zcs2s\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042311    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-var-lib-kubelet\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042337    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6634d566-76cf-4c96-9668-bc6ee975ed83-kubelet-config\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042377    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysctl-d\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042377    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-sys\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042399    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042409    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-var-lib-kubelet\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042418    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6634d566-76cf-4c96-9668-bc6ee975ed83-kubelet-config\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042426    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-host\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042491    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-host\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042514    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f912844-4661-427c-aa0a-139a17222d34-konnectivity-ca\") pod \"konnectivity-agent-nlz49\" (UID: \"7f912844-4661-427c-aa0a-139a17222d34\") " pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.042521    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:27.042555 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042557    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysctl-d\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.042598    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs podName:24645aa9-1df4-4033-84e5-00059d19bf6d nodeName:}" failed. No retries permitted until 2026-07-24 23:29:27.542575858 +0000 UTC m=+2.038287711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs") pod "network-metrics-daemon-wvlgk" (UID: "24645aa9-1df4-4033-84e5-00059d19bf6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042618    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgvhm\" (UniqueName: \"kubernetes.io/projected/24645aa9-1df4-4033-84e5-00059d19bf6d-kube-api-access-pgvhm\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042644    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6634d566-76cf-4c96-9668-bc6ee975ed83-dbus\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042690    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysctl-conf\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042708    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-245k9\" (UniqueName: \"kubernetes.io/projected/c3d9d2d2-b041-4df6-b863-40cd44584799-kube-api-access-245k9\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042760    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6634d566-76cf-4c96-9668-bc6ee975ed83-dbus\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042791    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b76e78ba-3363-4b80-9ac2-0ff25f65088f-serviceca\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042814    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f912844-4661-427c-aa0a-139a17222d34-agent-certs\") pod \"konnectivity-agent-nlz49\" (UID: \"7f912844-4661-427c-aa0a-139a17222d34\") " pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042846    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-systemd\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042871    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-modprobe-d\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042926    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-tuned\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042974    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysconfig\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.042999    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-lib-modules\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043031    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3d9d2d2-b041-4df6-b863-40cd44584799-tmp\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043035    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-systemd\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.043794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043043    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysctl-conf\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043074    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-kubernetes\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043099    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-sysconfig\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043103    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.043042    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043123    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f912844-4661-427c-aa0a-139a17222d34-konnectivity-ca\") pod \"konnectivity-agent-nlz49\" (UID: \"7f912844-4661-427c-aa0a-139a17222d34\") " pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043135    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-run\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.043157    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret podName:6634d566-76cf-4c96-9668-bc6ee975ed83 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:27.543141788 +0000 UTC m=+2.038853643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret") pod "global-pull-secret-syncer-n6fg5" (UID: "6634d566-76cf-4c96-9668-bc6ee975ed83") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043172    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-run\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043195    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b76e78ba-3363-4b80-9ac2-0ff25f65088f-host\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043200    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-lib-modules\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043213    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-kubernetes\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043218    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-modprobe-d\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043260    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b76e78ba-3363-4b80-9ac2-0ff25f65088f-host\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:27.044728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.043263    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b76e78ba-3363-4b80-9ac2-0ff25f65088f-serviceca\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:27.045298 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.045158    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c3d9d2d2-b041-4df6-b863-40cd44584799-etc-tuned\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.045298 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.045203    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3d9d2d2-b041-4df6-b863-40cd44584799-tmp\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.045494 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.045477    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f912844-4661-427c-aa0a-139a17222d34-agent-certs\") pod \"konnectivity-agent-nlz49\" (UID: \"7f912844-4661-427c-aa0a-139a17222d34\") " pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:27.049341 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.049322    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:27.049420 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.049343    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:27.049420 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.049353    2572 projected.go:196] Error preparing data for projected volume kube-api-access-5tvxk for pod openshift-network-diagnostics/network-check-target-cqzrz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:27.049539 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.049421    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk podName:4cd16aae-7f7b-4636-bd97-bec54f284e0f nodeName:}" failed. No retries permitted until 2026-07-24 23:29:27.549401483 +0000 UTC m=+2.045113350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5tvxk" (UniqueName: "kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk") pod "network-check-target-cqzrz" (UID: "4cd16aae-7f7b-4636-bd97-bec54f284e0f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:27.051127 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.051105    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-245k9\" (UniqueName: \"kubernetes.io/projected/c3d9d2d2-b041-4df6-b863-40cd44584799-kube-api-access-245k9\") pod \"tuned-nwsbv\" (UID: \"c3d9d2d2-b041-4df6-b863-40cd44584799\") " pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.051435 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.051415    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgvhm\" (UniqueName: \"kubernetes.io/projected/24645aa9-1df4-4033-84e5-00059d19bf6d-kube-api-access-pgvhm\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:27.051494 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.051476    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcs2s\" (UniqueName: \"kubernetes.io/projected/b76e78ba-3363-4b80-9ac2-0ff25f65088f-kube-api-access-zcs2s\") pod \"node-ca-6fwbl\" (UID: \"b76e78ba-3363-4b80-9ac2-0ff25f65088f\") " pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:27.143652 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.143617    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49"
Jul 24 23:29:27.153348 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.153306    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf983c8d_2cd0_4bfc_b91a_215765ffebcb.slice/crio-02f5f16a4a266207eb51cb5017e9fbf1cae120227a312bf089512f4f4a31066a WatchSource:0}: Error finding container 02f5f16a4a266207eb51cb5017e9fbf1cae120227a312bf089512f4f4a31066a: Status 404 returned error can't find the container with id 02f5f16a4a266207eb51cb5017e9fbf1cae120227a312bf089512f4f4a31066a
Jul 24 23:29:27.160477 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.160441    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c427z"
Jul 24 23:29:27.169618 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.169591    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608f5875_7843_48a2_a100_6050a0fdac6c.slice/crio-1a916e14c3df1cb889ad39d2daae09b2ea43acc14015ed72bbe71385227413bd WatchSource:0}: Error finding container 1a916e14c3df1cb889ad39d2daae09b2ea43acc14015ed72bbe71385227413bd: Status 404 returned error can't find the container with id 1a916e14c3df1cb889ad39d2daae09b2ea43acc14015ed72bbe71385227413bd
Jul 24 23:29:27.177776 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.177753    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdcv2"
Jul 24 23:29:27.186497 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.186442    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cac730_5c82_4f69_a783_34f405316b30.slice/crio-525a68e5c10592ea18be96b635458c275984e9041ef4f65943161eae82964239 WatchSource:0}: Error finding container 525a68e5c10592ea18be96b635458c275984e9041ef4f65943161eae82964239: Status 404 returned error can't find the container with id 525a68e5c10592ea18be96b635458c275984e9041ef4f65943161eae82964239
Jul 24 23:29:27.192059 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.192033    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zlbng"
Jul 24 23:29:27.200232 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.200207    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b33f0ce_edef_42b5_a87f_da504d0c0862.slice/crio-f71fa8633108fd76c1869f99003095add69a501ce4b110621725b67cf7fef6ac WatchSource:0}: Error finding container f71fa8633108fd76c1869f99003095add69a501ce4b110621725b67cf7fef6ac: Status 404 returned error can't find the container with id f71fa8633108fd76c1869f99003095add69a501ce4b110621725b67cf7fef6ac
Jul 24 23:29:27.214879 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.214847    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-72z7r"
Jul 24 23:29:27.223590 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.223561    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e1a018_5f68_4349_9b9b_7b6c3d267c53.slice/crio-9c45718a5b6c2b4ed4ea035619589fcefc3cd05b6627a64356f49477dd36f3a3 WatchSource:0}: Error finding container 9c45718a5b6c2b4ed4ea035619589fcefc3cd05b6627a64356f49477dd36f3a3: Status 404 returned error can't find the container with id 9c45718a5b6c2b4ed4ea035619589fcefc3cd05b6627a64356f49477dd36f3a3
Jul 24 23:29:27.234492 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.234443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:27.240932 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.240904    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:27.243738 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.243683    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a9e9b4_b3a1_454a_9e5d_cdda47a9540b.slice/crio-cebfea569e4f40683e6c29cca05da708e13173483dce36c6db56bab760fbb280 WatchSource:0}: Error finding container cebfea569e4f40683e6c29cca05da708e13173483dce36c6db56bab760fbb280: Status 404 returned error can't find the container with id cebfea569e4f40683e6c29cca05da708e13173483dce36c6db56bab760fbb280
Jul 24 23:29:27.250054 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.250028    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f912844_4661_427c_aa0a_139a17222d34.slice/crio-e65949cefb54ea056ab386fbe11024d74275796ce0ce084b5355b2e6b57ca3e5 WatchSource:0}: Error finding container e65949cefb54ea056ab386fbe11024d74275796ce0ce084b5355b2e6b57ca3e5: Status 404 returned error can't find the container with id e65949cefb54ea056ab386fbe11024d74275796ce0ce084b5355b2e6b57ca3e5
Jul 24 23:29:27.271445 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.271406    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nwsbv"
Jul 24 23:29:27.276360 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.276343    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6fwbl"
Jul 24 23:29:27.280734 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.280708    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d9d2d2_b041_4df6_b863_40cd44584799.slice/crio-269a8c44a4cc17e4239cbeca08f6bb3288e7afa961d5baa039d5afbccd997b62 WatchSource:0}: Error finding container 269a8c44a4cc17e4239cbeca08f6bb3288e7afa961d5baa039d5afbccd997b62: Status 404 returned error can't find the container with id 269a8c44a4cc17e4239cbeca08f6bb3288e7afa961d5baa039d5afbccd997b62
Jul 24 23:29:27.286124 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:29:27.286090    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76e78ba_3363_4b80_9ac2_0ff25f65088f.slice/crio-7a8960c7ba582efcffad08475c1a4c80668dd2beb1dc63ff17694b2e0302ba9d WatchSource:0}: Error finding container 7a8960c7ba582efcffad08475c1a4c80668dd2beb1dc63ff17694b2e0302ba9d: Status 404 returned error can't find the container with id 7a8960c7ba582efcffad08475c1a4c80668dd2beb1dc63ff17694b2e0302ba9d
Jul 24 23:29:27.546699 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.546357    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:27.546699 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.546423    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:27.546699 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.546527    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:27.546699 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.546566    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:27.546699 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.546594    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs podName:24645aa9-1df4-4033-84e5-00059d19bf6d nodeName:}" failed. No retries permitted until 2026-07-24 23:29:28.546573716 +0000 UTC m=+3.042285577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs") pod "network-metrics-daemon-wvlgk" (UID: "24645aa9-1df4-4033-84e5-00059d19bf6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:27.546699 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.546620    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret podName:6634d566-76cf-4c96-9668-bc6ee975ed83 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:28.546604148 +0000 UTC m=+3.042316004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret") pod "global-pull-secret-syncer-n6fg5" (UID: "6634d566-76cf-4c96-9668-bc6ee975ed83") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:27.647948 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.647913    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:27.648128 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.648069    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:27.648128 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.648087    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:27.648128 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.648099    2572 projected.go:196] Error preparing data for projected volume kube-api-access-5tvxk for pod openshift-network-diagnostics/network-check-target-cqzrz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:27.648290 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:27.648156    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk podName:4cd16aae-7f7b-4636-bd97-bec54f284e0f nodeName:}" failed. No retries permitted until 2026-07-24 23:29:28.648138734 +0000 UTC m=+3.143850590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tvxk" (UniqueName: "kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk") pod "network-check-target-cqzrz" (UID: "4cd16aae-7f7b-4636-bd97-bec54f284e0f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:27.693507 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.693474    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:27.875913 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.875748    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 23:24:26 +0000 UTC" deadline="2028-04-08 13:21:47.105528394 +0000 UTC"
Jul 24 23:29:27.875913 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.875815    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14965h52m19.229717803s"
Jul 24 23:29:27.981597 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:27.981570    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:28.011164 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.011098    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-72z7r" event={"ID":"92e1a018-5f68-4349-9b9b-7b6c3d267c53","Type":"ContainerStarted","Data":"9c45718a5b6c2b4ed4ea035619589fcefc3cd05b6627a64356f49477dd36f3a3"}
Jul 24 23:29:28.022757 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.022712    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zlbng" event={"ID":"1b33f0ce-edef-42b5-a87f-da504d0c0862","Type":"ContainerStarted","Data":"f71fa8633108fd76c1869f99003095add69a501ce4b110621725b67cf7fef6ac"}
Jul 24 23:29:28.039731 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.039689    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerStarted","Data":"525a68e5c10592ea18be96b635458c275984e9041ef4f65943161eae82964239"}
Jul 24 23:29:28.065124 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.065088    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49" event={"ID":"bf983c8d-2cd0-4bfc-b91a-215765ffebcb","Type":"ContainerStarted","Data":"02f5f16a4a266207eb51cb5017e9fbf1cae120227a312bf089512f4f4a31066a"}
Jul 24 23:29:28.091533 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.091428    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal" event={"ID":"690e7b1c9b542cf29dc8b6b0d7b228e9","Type":"ContainerStarted","Data":"24900b8c97fbbec928e0f136852aff8b0e83d5c0802b31f5f60c7078ef29938d"}
Jul 24 23:29:28.095005 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.094971    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nwsbv" event={"ID":"c3d9d2d2-b041-4df6-b863-40cd44584799","Type":"ContainerStarted","Data":"269a8c44a4cc17e4239cbeca08f6bb3288e7afa961d5baa039d5afbccd997b62"}
Jul 24 23:29:28.132841 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.132647    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"cebfea569e4f40683e6c29cca05da708e13173483dce36c6db56bab760fbb280"}
Jul 24 23:29:28.163520 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.162987    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c427z" event={"ID":"608f5875-7843-48a2-a100-6050a0fdac6c","Type":"ContainerStarted","Data":"1a916e14c3df1cb889ad39d2daae09b2ea43acc14015ed72bbe71385227413bd"}
Jul 24 23:29:28.182915 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.182867    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" event={"ID":"7c14bfa1a5ff22178fc8fb6624f7a3b8","Type":"ContainerStarted","Data":"151ff07976d2018198d2f5913691a25458d55a41a45b9ce10d18457c0fbe4d9b"}
Jul 24 23:29:28.215626 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.215582    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6fwbl" event={"ID":"b76e78ba-3363-4b80-9ac2-0ff25f65088f","Type":"ContainerStarted","Data":"7a8960c7ba582efcffad08475c1a4c80668dd2beb1dc63ff17694b2e0302ba9d"}
Jul 24 23:29:28.236542 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.236498    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nlz49" event={"ID":"7f912844-4661-427c-aa0a-139a17222d34","Type":"ContainerStarted","Data":"e65949cefb54ea056ab386fbe11024d74275796ce0ce084b5355b2e6b57ca3e5"}
Jul 24 23:29:28.556202 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.556114    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:28.556202 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.556177    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:28.556418 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.556348    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:28.556418 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.556410    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret podName:6634d566-76cf-4c96-9668-bc6ee975ed83 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:30.556391101 +0000 UTC m=+5.052102956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret") pod "global-pull-secret-syncer-n6fg5" (UID: "6634d566-76cf-4c96-9668-bc6ee975ed83") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:28.556842 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.556820    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:28.556920 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.556880    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs podName:24645aa9-1df4-4033-84e5-00059d19bf6d nodeName:}" failed. No retries permitted until 2026-07-24 23:29:30.556864379 +0000 UTC m=+5.052576249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs") pod "network-metrics-daemon-wvlgk" (UID: "24645aa9-1df4-4033-84e5-00059d19bf6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:28.657347 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.656763    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:28.657347 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.656927    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:28.657347 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.656945    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:28.657347 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.656959    2572 projected.go:196] Error preparing data for projected volume kube-api-access-5tvxk for pod openshift-network-diagnostics/network-check-target-cqzrz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:28.657347 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.657013    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk podName:4cd16aae-7f7b-4636-bd97-bec54f284e0f nodeName:}" failed. No retries permitted until 2026-07-24 23:29:30.656996002 +0000 UTC m=+5.152707859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tvxk" (UniqueName: "kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk") pod "network-check-target-cqzrz" (UID: "4cd16aae-7f7b-4636-bd97-bec54f284e0f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:28.876794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.876706    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 23:24:26 +0000 UTC" deadline="2028-02-29 22:59:34.336101119 +0000 UTC"
Jul 24 23:29:28.876794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.876744    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14039h30m5.459361903s"
Jul 24 23:29:28.897372 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.897341    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:28.960532 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.960497    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:28.960716 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.960627    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:28.960779 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.960712    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:28.960779 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.960625    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:28.960878 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:28.960788    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:28.960878 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:28.960853    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:30.573350 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:30.573313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:30.573857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:30.573414    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:30.573857 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.573565    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:30.573857 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.573635    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs podName:24645aa9-1df4-4033-84e5-00059d19bf6d nodeName:}" failed. No retries permitted until 2026-07-24 23:29:34.573617406 +0000 UTC m=+9.069329264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs") pod "network-metrics-daemon-wvlgk" (UID: "24645aa9-1df4-4033-84e5-00059d19bf6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:30.574051 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.574032    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:30.574103 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.574085    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret podName:6634d566-76cf-4c96-9668-bc6ee975ed83 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:34.574069421 +0000 UTC m=+9.069781276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret") pod "global-pull-secret-syncer-n6fg5" (UID: "6634d566-76cf-4c96-9668-bc6ee975ed83") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:30.674126 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:30.674037    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:30.674308 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.674213    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:30.674308 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.674233    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:30.674308 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.674247    2572 projected.go:196] Error preparing data for projected volume kube-api-access-5tvxk for pod openshift-network-diagnostics/network-check-target-cqzrz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:30.674516 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.674313    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk podName:4cd16aae-7f7b-4636-bd97-bec54f284e0f nodeName:}" failed. No retries permitted until 2026-07-24 23:29:34.674296883 +0000 UTC m=+9.170008755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tvxk" (UniqueName: "kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk") pod "network-check-target-cqzrz" (UID: "4cd16aae-7f7b-4636-bd97-bec54f284e0f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:30.961676 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:30.960763    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:30.961676 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.960906    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:30.961676 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:30.961287    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:30.961676 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.961375    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:30.961676 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:30.961435    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:30.961676 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:30.961531    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:32.961070 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:32.960931    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:32.961070 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:32.960986    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:32.961070 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:32.960926    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:32.962132 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:32.961128    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:32.962132 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:32.961645    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:32.962132 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:32.961760    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:34.610439 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:34.609844    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:34.610439 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:34.609903    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:34.610439 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.610041    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:34.610439 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.610116    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret podName:6634d566-76cf-4c96-9668-bc6ee975ed83 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:42.610093898 +0000 UTC m=+17.105805774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret") pod "global-pull-secret-syncer-n6fg5" (UID: "6634d566-76cf-4c96-9668-bc6ee975ed83") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:34.610439 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.610133    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:34.610439 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.610172    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs podName:24645aa9-1df4-4033-84e5-00059d19bf6d nodeName:}" failed. No retries permitted until 2026-07-24 23:29:42.610160851 +0000 UTC m=+17.105872717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs") pod "network-metrics-daemon-wvlgk" (UID: "24645aa9-1df4-4033-84e5-00059d19bf6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:34.710990 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:34.710953    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:34.711190 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.711137    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:34.711190 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.711160    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:34.711190 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.711172    2572 projected.go:196] Error preparing data for projected volume kube-api-access-5tvxk for pod openshift-network-diagnostics/network-check-target-cqzrz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:34.711384 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.711234    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk podName:4cd16aae-7f7b-4636-bd97-bec54f284e0f nodeName:}" failed. No retries permitted until 2026-07-24 23:29:42.711214901 +0000 UTC m=+17.206926757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tvxk" (UniqueName: "kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk") pod "network-check-target-cqzrz" (UID: "4cd16aae-7f7b-4636-bd97-bec54f284e0f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:34.961441 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:34.961354    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:34.961616 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.961508    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:34.961925 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:34.961896    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:34.962034 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.962015    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:34.962102 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:34.962061    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:34.962160 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:34.962127    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:36.961316 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:36.961227    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:36.961316 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:36.961257    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:36.961789 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:36.961385    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:36.961789 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:36.961535    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:36.961789 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:36.961614    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:36.961789 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:36.961690    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:38.961162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:38.961121    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:38.961542 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:38.961165    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:38.961542 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:38.961131    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:38.961542 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:38.961240    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:38.961542 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:38.961303    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:38.961542 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:38.961384    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:40.961378 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:40.961333    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:40.961378 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:40.961366    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:40.961378 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:40.961376    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:40.961965 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:40.961487    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:40.961965 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:40.961562    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:40.961965 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:40.961656    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:42.674911 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:42.674856    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:42.675432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:42.674926    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:42.675432 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.675028    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:42.675432 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.675087    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret podName:6634d566-76cf-4c96-9668-bc6ee975ed83 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:58.675070548 +0000 UTC m=+33.170782403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret") pod "global-pull-secret-syncer-n6fg5" (UID: "6634d566-76cf-4c96-9668-bc6ee975ed83") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:42.675432 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.675029    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:42.675432 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.675172    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs podName:24645aa9-1df4-4033-84e5-00059d19bf6d nodeName:}" failed. No retries permitted until 2026-07-24 23:29:58.675151318 +0000 UTC m=+33.170863174 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs") pod "network-metrics-daemon-wvlgk" (UID: "24645aa9-1df4-4033-84e5-00059d19bf6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:42.775583 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:42.775533    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:42.775785 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.775722    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:42.775785 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.775755    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:42.775785 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.775768    2572 projected.go:196] Error preparing data for projected volume kube-api-access-5tvxk for pod openshift-network-diagnostics/network-check-target-cqzrz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:42.775918 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.775822    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk podName:4cd16aae-7f7b-4636-bd97-bec54f284e0f nodeName:}" failed. No retries permitted until 2026-07-24 23:29:58.775807841 +0000 UTC m=+33.271519694 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tvxk" (UniqueName: "kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk") pod "network-check-target-cqzrz" (UID: "4cd16aae-7f7b-4636-bd97-bec54f284e0f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:42.961331 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:42.961241    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:42.961331 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:42.961304    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:42.961628 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:42.961429    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:42.961628 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.961437    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:42.961628 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.961555    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:42.961766 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:42.961662    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:44.961426 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:44.961222    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:44.961912 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:44.961249    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:44.961912 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:44.961525    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:44.961912 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:44.961258    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:44.961912 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:44.961592    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:44.961912 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:44.961674    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:46.961276 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:46.961237    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:46.961276 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:46.961259    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:46.961678 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:46.961283    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:46.961678 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:46.961389    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:46.961678 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:46.961501    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:46.961678 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:46.961569    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:47.298857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:47.298674    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal" event={"ID":"690e7b1c9b542cf29dc8b6b0d7b228e9","Type":"ContainerStarted","Data":"436e132f7e13ec665b7ea7feec1109792a5e3bffd175a7cfaf52a3d3c302d515"}
Jul 24 23:29:47.310067 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:47.310018    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nwsbv" event={"ID":"c3d9d2d2-b041-4df6-b863-40cd44584799","Type":"ContainerStarted","Data":"b14497704e4deec2e2eba2f5f85794c4f1f7c635bb677d95512938284f765a19"}
Jul 24 23:29:47.312100 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:47.311829    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal" podStartSLOduration=21.311814209 podStartE2EDuration="21.311814209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:47.311551178 +0000 UTC m=+21.807263051" watchObservedRunningTime="2026-07-24 23:29:47.311814209 +0000 UTC m=+21.807526084"
Jul 24 23:29:47.322027 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:47.321982    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"c996f86cb4f780c1ee4e049b334a4b190bcf1f97e70e7b7beb5dbc4f330ad22a"}
Jul 24 23:29:47.329599 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:47.329567    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zlbng" event={"ID":"1b33f0ce-edef-42b5-a87f-da504d0c0862","Type":"ContainerStarted","Data":"4f70b73039c408e98798b3949219677bb291a676ef20fbd830bb5cb30d991f50"}
Jul 24 23:29:47.347841 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:47.347774    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zlbng" podStartSLOduration=1.374817729 podStartE2EDuration="21.347750956s" totalImagesPullingTime="19.972933227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:47.34772561 +0000 UTC m=+21.843437486" watchObservedRunningTime="2026-07-24 23:29:47.347750956 +0000 UTC m=+21.843462830"
Jul 24 23:29:47.348599 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:47.348346    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nwsbv" podStartSLOduration=1.60590992 podStartE2EDuration="21.348336637s" totalImagesPullingTime="19.742426717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:47.326210876 +0000 UTC m=+21.821922751" watchObservedRunningTime="2026-07-24 23:29:47.348336637 +0000 UTC m=+21.844048513"
Jul 24 23:29:48.333519 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.333414    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nlz49" event={"ID":"7f912844-4661-427c-aa0a-139a17222d34","Type":"ContainerStarted","Data":"881c4390e2eba398c15a8094e837848ea7c04882e1249989bf4c38ff666a1a33"}
Jul 24 23:29:48.335090 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.335065    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4cac730-5c82-4f69-a783-34f405316b30" containerID="d1aabe3429894630590451e60ebca6bf93403156a64da0e1234e5231bd185383" exitCode=0
Jul 24 23:29:48.335172 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.335141    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerDied","Data":"d1aabe3429894630590451e60ebca6bf93403156a64da0e1234e5231bd185383"}
Jul 24 23:29:48.336707 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.336642    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49" event={"ID":"bf983c8d-2cd0-4bfc-b91a-215765ffebcb","Type":"ContainerStarted","Data":"9bb613f9257e29399634dd7ba97517c9896bf6ac6ffa5ae3456af4d24c5f0996"}
Jul 24 23:29:48.339786 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.339763    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"5de0ee383475aa88e71ffade91aa65dcd1799f715cb9d0c442c70d94eccb382b"}
Jul 24 23:29:48.339891 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.339843    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"0875fafa0c2ddf1e7f98404f585fb266746bde458edb80e3cd81e25995920a65"}
Jul 24 23:29:48.339934 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.339902    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"1e02885e903af6d0ffeac37357c0aac898ada11c87cf412abe4175166eae1792"}
Jul 24 23:29:48.339980 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.339954    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"6942f1af3c5d9ae6233248fa96019c8ed5180ff74c8e54b5a2c974a4e3bc89d7"}
Jul 24 23:29:48.340020 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.340001    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"a090e54e10afffc53145a3a955f5ec206a3f5207a8766215da0ecfec70b08500"}
Jul 24 23:29:48.341193 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.341170    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c427z" event={"ID":"608f5875-7843-48a2-a100-6050a0fdac6c","Type":"ContainerStarted","Data":"1170dac39a8ca74422f6015122407e49bd4f40f86645e7b0326f709c4c4c633d"}
Jul 24 23:29:48.342602 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.342579    2572 generic.go:356] "Generic (PLEG): container finished" podID="7c14bfa1a5ff22178fc8fb6624f7a3b8" containerID="d7cb1c390099cde0010d4eabf7d0a49ffdd2af206c43277b12aa72c5fe789d91" exitCode=0
Jul 24 23:29:48.342699 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.342661    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" event={"ID":"7c14bfa1a5ff22178fc8fb6624f7a3b8","Type":"ContainerDied","Data":"d7cb1c390099cde0010d4eabf7d0a49ffdd2af206c43277b12aa72c5fe789d91"}
Jul 24 23:29:48.344070 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.344048    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6fwbl" event={"ID":"b76e78ba-3363-4b80-9ac2-0ff25f65088f","Type":"ContainerStarted","Data":"99a39dafbabf1e1986ef8f4319b0239c0e56f107a7476189cd99d6d2c34d5b3f"}
Jul 24 23:29:48.361631 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.361576    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nlz49" podStartSLOduration=2.59102607 podStartE2EDuration="22.361556594s" totalImagesPullingTime="19.770530524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:48.346531958 +0000 UTC m=+22.842243830" watchObservedRunningTime="2026-07-24 23:29:48.361556594 +0000 UTC m=+22.857268470"
Jul 24 23:29:48.373787 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.373743    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-c427z" podStartSLOduration=2.553070569 podStartE2EDuration="22.373726751s" totalImagesPullingTime="19.820656182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:48.373517622 +0000 UTC m=+22.869229497" watchObservedRunningTime="2026-07-24 23:29:48.373726751 +0000 UTC m=+22.869438626"
Jul 24 23:29:48.386234 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.385978    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6fwbl" podStartSLOduration=2.6517032350000003 podStartE2EDuration="22.385960154s" totalImagesPullingTime="19.734256919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:48.385689333 +0000 UTC m=+22.881401207" watchObservedRunningTime="2026-07-24 23:29:48.385960154 +0000 UTC m=+22.881672030"
Jul 24 23:29:48.602101 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.602071    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 23:29:48.914711 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.914543    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T23:29:48.602092211Z","UUID":"24e3bf67-b6c7-49e1-a083-8c858a91a679","Handler":null,"Name":"","Endpoint":""}
Jul 24 23:29:48.916200 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.916171    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 23:29:48.916345 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.916214    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 23:29:48.961313 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.961280    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:48.961547 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.961410    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:48.961547 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:48.961421    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:48.961547 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:48.961413    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:48.961547 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:48.961536    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:48.961758 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:48.961643    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:49.248669 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.248590    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:49.249412 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.249392    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:49.350517 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.350477    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" event={"ID":"7c14bfa1a5ff22178fc8fb6624f7a3b8","Type":"ContainerStarted","Data":"6591694751a197d0da37003a674de85972c747b0cff6d9ba034a5684144de933"}
Jul 24 23:29:49.352299 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.352258    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-72z7r" event={"ID":"92e1a018-5f68-4349-9b9b-7b6c3d267c53","Type":"ContainerStarted","Data":"c5e0153218dfd546eabd0feb47fa3f06608a5378a36c79f05002b31504fc4233"}
Jul 24 23:29:49.354776 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.354748    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49" event={"ID":"bf983c8d-2cd0-4bfc-b91a-215765ffebcb","Type":"ContainerStarted","Data":"ff5e1a29a9e023b723a1a53483142e8655ab3d5c63a7159cbf7a70a0c2f1cb1a"}
Jul 24 23:29:49.355338 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.355313    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:49.355960 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.355933    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nlz49"
Jul 24 23:29:49.366318 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.366243    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" podStartSLOduration=23.366224087 podStartE2EDuration="23.366224087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:49.365939114 +0000 UTC m=+23.861650990" watchObservedRunningTime="2026-07-24 23:29:49.366224087 +0000 UTC m=+23.861935965"
Jul 24 23:29:49.381856 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:49.381806    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-72z7r" podStartSLOduration=3.615112353 podStartE2EDuration="23.381785486s" totalImagesPullingTime="19.766673133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:49.381423448 +0000 UTC m=+23.877135336" watchObservedRunningTime="2026-07-24 23:29:49.381785486 +0000 UTC m=+23.877497362"
Jul 24 23:29:50.360267 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:50.360225    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49" event={"ID":"bf983c8d-2cd0-4bfc-b91a-215765ffebcb","Type":"ContainerStarted","Data":"151e0bdbc631e19f18d418534e3553b19c5b32e244d7031ec9475b1f54ffe1c8"}
Jul 24 23:29:50.364363 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:50.364328    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"0475172dc100f1ee5b128c64d400f28663aad342031c5342069e2f7f6f43e8cd"}
Jul 24 23:29:50.377486 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:50.377416    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzp49" podStartSLOduration=2.376400008 podStartE2EDuration="24.377394884s" totalImagesPullingTime="22.000994876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:50.376478864 +0000 UTC m=+24.872190738" watchObservedRunningTime="2026-07-24 23:29:50.377394884 +0000 UTC m=+24.873106854"
Jul 24 23:29:50.961344 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:50.961301    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:50.961344 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:50.961327    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:50.961656 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:50.961357    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:50.961656 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:50.961437    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:50.961656 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:50.961504    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:50.961656 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:50.961578    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:52.960779 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:52.960539    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:52.961279 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:52.960539    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:52.961279 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:52.960811    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:52.961279 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:52.960539    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:52.961279 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:52.960879    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:52.961279 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:52.960948    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:53.374587 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.374495    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" event={"ID":"b1a9e9b4-b3a1-454a-9e5d-cdda47a9540b","Type":"ContainerStarted","Data":"6baf1a03182527b436c0d0dcce558c383bb12f4b52f3dca0c190e9b99b6d4a26"}
Jul 24 23:29:53.374897 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.374745    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:53.374897 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.374817    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:53.374897 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.374863    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:53.376685 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.376654    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4cac730-5c82-4f69-a783-34f405316b30" containerID="59a7e45e1ae2a1503a710905888eb022be13244572740302cc7c6fb77370aefa" exitCode=0
Jul 24 23:29:53.376813 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.376694    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerDied","Data":"59a7e45e1ae2a1503a710905888eb022be13244572740302cc7c6fb77370aefa"}
Jul 24 23:29:53.391472 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.391428    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:53.391634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.391552    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:29:53.399174 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:53.399126    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg" podStartSLOduration=7.600767542 podStartE2EDuration="27.399111872s" totalImagesPullingTime="19.79834433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:53.397754711 +0000 UTC m=+27.893466586" watchObservedRunningTime="2026-07-24 23:29:53.399111872 +0000 UTC m=+27.894823772"
Jul 24 23:29:54.960557 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:54.960512    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:54.960557 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:54.960566    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:54.961199 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:54.960512    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:54.961199 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:54.960655    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:54.961199 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:54.960761    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:54.961199 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:54.960831    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:55.383637 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:55.383601    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4cac730-5c82-4f69-a783-34f405316b30" containerID="c4c61cf901d8f9fdad0789024c2d2593533ce978ca84ee0059fecc0e62ff10fb" exitCode=0
Jul 24 23:29:55.383805 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:55.383696    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerDied","Data":"c4c61cf901d8f9fdad0789024c2d2593533ce978ca84ee0059fecc0e62ff10fb"}
Jul 24 23:29:56.960523 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:56.960482    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:56.960906 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:56.960482    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:56.960906 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:56.960624    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:56.960906 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:56.960482    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:56.960906 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:56.960662    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:56.960906 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:56.960751    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:29:57.390579 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:57.390488    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4cac730-5c82-4f69-a783-34f405316b30" containerID="d6b1392a2416620ebc8a6f1ca8cdde631be7614c7c5f19abc16368f54950c551" exitCode=0
Jul 24 23:29:57.390579 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:57.390551    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerDied","Data":"d6b1392a2416620ebc8a6f1ca8cdde631be7614c7c5f19abc16368f54950c551"}
Jul 24 23:29:58.689640 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:58.689599    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:58.690078 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:58.689656    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:58.690078 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.689778    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:58.690078 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.689785    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:58.690078 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.689858    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret podName:6634d566-76cf-4c96-9668-bc6ee975ed83 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:30.68983753 +0000 UTC m=+65.185549383 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret") pod "global-pull-secret-syncer-n6fg5" (UID: "6634d566-76cf-4c96-9668-bc6ee975ed83") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:58.690078 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.689879    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs podName:24645aa9-1df4-4033-84e5-00059d19bf6d nodeName:}" failed. No retries permitted until 2026-07-24 23:30:30.68986888 +0000 UTC m=+65.185580736 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs") pod "network-metrics-daemon-wvlgk" (UID: "24645aa9-1df4-4033-84e5-00059d19bf6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:58.790316 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:58.790277    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:58.790536 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.790485    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:58.790536 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.790510    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:58.790536 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.790524    2572 projected.go:196] Error preparing data for projected volume kube-api-access-5tvxk for pod openshift-network-diagnostics/network-check-target-cqzrz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:58.790704 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.790592    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk podName:4cd16aae-7f7b-4636-bd97-bec54f284e0f nodeName:}" failed. No retries permitted until 2026-07-24 23:30:30.790573432 +0000 UTC m=+65.286285289 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tvxk" (UniqueName: "kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk") pod "network-check-target-cqzrz" (UID: "4cd16aae-7f7b-4636-bd97-bec54f284e0f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:58.961179 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:58.961091    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:29:58.961338 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:58.961102    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:29:58.961338 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.961221    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:29:58.961338 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:29:58.961232    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:29:58.961338 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.961312    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:29:58.961553 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:29:58.961407    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:00.960726 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:00.960691    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:00.961146 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:00.960799    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:00.961146 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:00.960806    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:00.961146 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:00.960808    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:00.961146 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:00.960901    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:00.961146 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:00.960969    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:01.404426 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:01.404227    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerStarted","Data":"63484121963a1c9ef5e64574291f6fe466d7f06aea0d6748181d3feae2115135"}
Jul 24 23:30:02.409159 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:02.409122    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4cac730-5c82-4f69-a783-34f405316b30" containerID="63484121963a1c9ef5e64574291f6fe466d7f06aea0d6748181d3feae2115135" exitCode=0
Jul 24 23:30:02.409565 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:02.409207    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerDied","Data":"63484121963a1c9ef5e64574291f6fe466d7f06aea0d6748181d3feae2115135"}
Jul 24 23:30:02.960611 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:02.960578    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:02.960797 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:02.960578    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:02.960797 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:02.960687    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:02.960797 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:02.960578    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:02.960797 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:02.960762    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:02.960919 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:02.960838    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:03.414619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:03.414585    2572 generic.go:356] "Generic (PLEG): container finished" podID="c4cac730-5c82-4f69-a783-34f405316b30" containerID="1dd0f36340285d693ba6749f280daf29c2844d9ad0c6945e43caaef495b07aae" exitCode=0
Jul 24 23:30:03.415025 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:03.414654    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerDied","Data":"1dd0f36340285d693ba6749f280daf29c2844d9ad0c6945e43caaef495b07aae"}
Jul 24 23:30:04.420208 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:04.420173    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" event={"ID":"c4cac730-5c82-4f69-a783-34f405316b30","Type":"ContainerStarted","Data":"f5e0e62b22b71430d08d24f0819221c2c82d374abff35ba90c660f138ab6dd40"}
Jul 24 23:30:04.441278 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:04.441220    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jdcv2" podStartSLOduration=8.413256455 podStartE2EDuration="38.441182227s" totalImagesPullingTime="30.027925772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:04.439794967 +0000 UTC m=+38.935506841" watchObservedRunningTime="2026-07-24 23:30:04.441182227 +0000 UTC m=+38.936894102"
Jul 24 23:30:04.961283 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:04.961241    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:04.961494 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:04.961241    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:04.961494 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:04.961369    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:04.961494 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:04.961241    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:04.961494 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:04.961430    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:04.961632 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:04.961537    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:06.960685 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:06.960644    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:06.960685 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:06.960677    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:06.961202 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:06.960684    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:06.961202 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:06.960813    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:06.961202 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:06.961132    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:06.961441 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:06.961416    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:08.961228 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:08.961179    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:08.961228 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:08.961206    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:08.961700 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:08.961180    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:08.961700 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:08.961312    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:08.961700 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:08.961376    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:08.961700 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:08.961437    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:09.615700 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:09.615624    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wvlgk"]
Jul 24 23:30:09.615889 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:09.615769    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:09.615965 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:09.615900    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:09.618829 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:09.618626    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n6fg5"]
Jul 24 23:30:09.618829 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:09.618798    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:09.619037 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:09.618912    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:09.619312 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:09.619291    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cqzrz"]
Jul 24 23:30:09.619399 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:09.619386    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:09.619521 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:09.619499    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:10.961093 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:10.960999    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:10.961527 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:10.960999    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:10.961527 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:10.961119    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:10.961527 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:10.961185    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:10.961527 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:10.961025    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:10.961527 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:10.961254    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:12.960871 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:12.960679    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:12.961267 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:12.960679    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:12.961267 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:12.960957    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wvlgk" podUID="24645aa9-1df4-4033-84e5-00059d19bf6d"
Jul 24 23:30:12.961267 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:12.960722    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:12.961267 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:12.961016    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6fg5" podUID="6634d566-76cf-4c96-9668-bc6ee975ed83"
Jul 24 23:30:12.961267 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:12.961089    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cqzrz" podUID="4cd16aae-7f7b-4636-bd97-bec54f284e0f"
Jul 24 23:30:13.809963 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.809938    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeReady"
Jul 24 23:30:13.810161 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.810088    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 23:30:13.851195 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.851161    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fg5dg"]
Jul 24 23:30:13.876996 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.876916    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-84pqs"]
Jul 24 23:30:13.877173 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.877046    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:13.879465 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.879423    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 23:30:13.879616 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.879561    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 23:30:13.879616 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.879572    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 23:30:13.879616 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.879581    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 23:30:13.879857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.879841    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkx27\""
Jul 24 23:30:13.901182 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.901144    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fg5dg"]
Jul 24 23:30:13.901182 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.901191    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84pqs"]
Jul 24 23:30:13.901379 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.901206    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dhmkj"]
Jul 24 23:30:13.901379 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.901251    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:13.903635 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.903615    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 23:30:13.903787 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.903703    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cqz6m\""
Jul 24 23:30:13.903787 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.903718    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 23:30:13.903787 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.903759    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 23:30:13.919356 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.919325    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dhmkj"]
Jul 24 23:30:13.919530 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.919469    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:13.921762 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.921741    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 23:30:13.921762 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.921758    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 23:30:13.921914 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:13.921760    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2kgw4\""
Jul 24 23:30:14.001418 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.001374    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-crio-socket\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.001418 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.001416    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgxjw\" (UniqueName: \"kubernetes.io/projected/648fb8e3-226a-422a-8e0b-cbdf0e58a310-kube-api-access-qgxjw\") pod \"ingress-canary-84pqs\" (UID: \"648fb8e3-226a-422a-8e0b-cbdf0e58a310\") " pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:14.001941 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.001439    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7ml\" (UniqueName: \"kubernetes.io/projected/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-kube-api-access-xh7ml\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.001941 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.001563    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.001941 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.001616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648fb8e3-226a-422a-8e0b-cbdf0e58a310-cert\") pod \"ingress-canary-84pqs\" (UID: \"648fb8e3-226a-422a-8e0b-cbdf0e58a310\") " pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:14.001941 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.001659    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-data-volume\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.001941 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.001740    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.102880 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.102840    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e929bc11-c049-490a-a60e-48be763e1b27-metrics-tls\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.102880 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.102881    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e929bc11-c049-490a-a60e-48be763e1b27-config-volume\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.103049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.102896    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e929bc11-c049-490a-a60e-48be763e1b27-tmp-dir\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.103049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.102969    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.103049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103018    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-crio-socket\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.103049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgxjw\" (UniqueName: \"kubernetes.io/projected/648fb8e3-226a-422a-8e0b-cbdf0e58a310-kube-api-access-qgxjw\") pod \"ingress-canary-84pqs\" (UID: \"648fb8e3-226a-422a-8e0b-cbdf0e58a310\") " pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:14.103241 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103072    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7ml\" (UniqueName: \"kubernetes.io/projected/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-kube-api-access-xh7ml\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.103241 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103122    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.103241 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103153    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njdzt\" (UniqueName: \"kubernetes.io/projected/e929bc11-c049-490a-a60e-48be763e1b27-kube-api-access-njdzt\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.103241 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103187    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648fb8e3-226a-422a-8e0b-cbdf0e58a310-cert\") pod \"ingress-canary-84pqs\" (UID: \"648fb8e3-226a-422a-8e0b-cbdf0e58a310\") " pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:14.103241 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103227    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-data-volume\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.103499 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103282    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-crio-socket\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.103660 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103641    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-data-volume\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.103728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.103702    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.107386 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.107360    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.107530 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.107410    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/648fb8e3-226a-422a-8e0b-cbdf0e58a310-cert\") pod \"ingress-canary-84pqs\" (UID: \"648fb8e3-226a-422a-8e0b-cbdf0e58a310\") " pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:14.110635 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.110607    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7ml\" (UniqueName: \"kubernetes.io/projected/46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f-kube-api-access-xh7ml\") pod \"insights-runtime-extractor-fg5dg\" (UID: \"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f\") " pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.110758 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.110679    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgxjw\" (UniqueName: \"kubernetes.io/projected/648fb8e3-226a-422a-8e0b-cbdf0e58a310-kube-api-access-qgxjw\") pod \"ingress-canary-84pqs\" (UID: \"648fb8e3-226a-422a-8e0b-cbdf0e58a310\") " pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:14.187758 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.187673    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fg5dg"
Jul 24 23:30:14.204008 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.203980    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njdzt\" (UniqueName: \"kubernetes.io/projected/e929bc11-c049-490a-a60e-48be763e1b27-kube-api-access-njdzt\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.204115 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.204033    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e929bc11-c049-490a-a60e-48be763e1b27-metrics-tls\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.204115 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.204055    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e929bc11-c049-490a-a60e-48be763e1b27-config-volume\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.204115 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.204071    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e929bc11-c049-490a-a60e-48be763e1b27-tmp-dir\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.204540 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.204511    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e929bc11-c049-490a-a60e-48be763e1b27-tmp-dir\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.204731 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.204711    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e929bc11-c049-490a-a60e-48be763e1b27-config-volume\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.206573 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.206547    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e929bc11-c049-490a-a60e-48be763e1b27-metrics-tls\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.209796 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.209765    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84pqs"
Jul 24 23:30:14.211862 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.211824    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njdzt\" (UniqueName: \"kubernetes.io/projected/e929bc11-c049-490a-a60e-48be763e1b27-kube-api-access-njdzt\") pod \"dns-default-dhmkj\" (UID: \"e929bc11-c049-490a-a60e-48be763e1b27\") " pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.227505 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.227473    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:14.399616 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.399584    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84pqs"]
Jul 24 23:30:14.402190 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.402156    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fg5dg"]
Jul 24 23:30:14.405303 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:14.405272    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b4d986_fc79_4c4d_ba2c_2b6e4f56ac0f.slice/crio-ea06770ee664ae94ea80681179712ea9bde2b400a09e4573b1d16bffd1df8103 WatchSource:0}: Error finding container ea06770ee664ae94ea80681179712ea9bde2b400a09e4573b1d16bffd1df8103: Status 404 returned error can't find the container with id ea06770ee664ae94ea80681179712ea9bde2b400a09e4573b1d16bffd1df8103
Jul 24 23:30:14.407492 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.407471    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dhmkj"]
Jul 24 23:30:14.410705 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:14.410678    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode929bc11_c049_490a_a60e_48be763e1b27.slice/crio-91adfdc94f576a94be04452fd4ded590a9c2e27dbcd4eb09494f654578480899 WatchSource:0}: Error finding container 91adfdc94f576a94be04452fd4ded590a9c2e27dbcd4eb09494f654578480899: Status 404 returned error can't find the container with id 91adfdc94f576a94be04452fd4ded590a9c2e27dbcd4eb09494f654578480899
Jul 24 23:30:14.444438 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.444336    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dhmkj" event={"ID":"e929bc11-c049-490a-a60e-48be763e1b27","Type":"ContainerStarted","Data":"91adfdc94f576a94be04452fd4ded590a9c2e27dbcd4eb09494f654578480899"}
Jul 24 23:30:14.445501 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.445477    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84pqs" event={"ID":"648fb8e3-226a-422a-8e0b-cbdf0e58a310","Type":"ContainerStarted","Data":"8142aa94a99ad25ce075d057100b5fb5d400b5bd8b547603c5eb7bd41a01c784"}
Jul 24 23:30:14.446433 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.446396    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fg5dg" event={"ID":"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f","Type":"ContainerStarted","Data":"ea06770ee664ae94ea80681179712ea9bde2b400a09e4573b1d16bffd1df8103"}
Jul 24 23:30:14.961325 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.961288    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:14.961763 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.961746    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:14.962613 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.962278    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:14.966367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.965350    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 23:30:14.966367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.965607    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 23:30:14.966367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.965843    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzdt8\""
Jul 24 23:30:14.966367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.966049    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 23:30:14.966367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.966221    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 23:30:14.966367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:14.966331    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t27k5\""
Jul 24 23:30:15.042511 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.042473    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64c755bfbd-sp4r9"]
Jul 24 23:30:15.055154 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.054918    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.058712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.056490    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c755bfbd-sp4r9"]
Jul 24 23:30:15.058712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058137    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 23:30:15.058712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058146    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 23:30:15.058712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058382    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 23:30:15.058712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058482    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 23:30:15.058712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058653    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 23:30:15.058712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058699    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 23:30:15.059183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058833    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 23:30:15.059183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.058921    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fprt5\""
Jul 24 23:30:15.210448 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.210407    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-oauth-config\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.210656 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.210495    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cchcj\" (UniqueName: \"kubernetes.io/projected/fbb6ab74-a15a-4830-af34-6ddf853c57df-kube-api-access-cchcj\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.210656 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.210525    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-config\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.210656 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.210555    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-serving-cert\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.210656 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.210612    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-oauth-serving-cert\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.210865 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.210656    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-service-ca\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.311280 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.311021    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-oauth-config\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.311535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.311323    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cchcj\" (UniqueName: \"kubernetes.io/projected/fbb6ab74-a15a-4830-af34-6ddf853c57df-kube-api-access-cchcj\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.311535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.311361    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-config\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.311535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.311393    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-serving-cert\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.311535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.311467    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-oauth-serving-cert\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.311535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.311507    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-service-ca\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.312388 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.312360    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-service-ca\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.312946 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.312920    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-oauth-serving-cert\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.313124 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.313103    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-config\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.316590 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.316538    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-serving-cert\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.318626 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.318599    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-oauth-config\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.321002 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.320966    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cchcj\" (UniqueName: \"kubernetes.io/projected/fbb6ab74-a15a-4830-af34-6ddf853c57df-kube-api-access-cchcj\") pod \"console-64c755bfbd-sp4r9\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") " pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.369859 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.369537    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:15.551509 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:15.551479    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c755bfbd-sp4r9"]
Jul 24 23:30:15.557919 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:15.557879    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb6ab74_a15a_4830_af34_6ddf853c57df.slice/crio-5a70df1da662619bf9af83f0e1005041c2819ef82e64537b2e7668fd5aca1e6f WatchSource:0}: Error finding container 5a70df1da662619bf9af83f0e1005041c2819ef82e64537b2e7668fd5aca1e6f: Status 404 returned error can't find the container with id 5a70df1da662619bf9af83f0e1005041c2819ef82e64537b2e7668fd5aca1e6f
Jul 24 23:30:16.458547 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.458519    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8664ff4bc4-5ffwz"]
Jul 24 23:30:16.474866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.474812    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c755bfbd-sp4r9" event={"ID":"fbb6ab74-a15a-4830-af34-6ddf853c57df","Type":"ContainerStarted","Data":"5a70df1da662619bf9af83f0e1005041c2819ef82e64537b2e7668fd5aca1e6f"}
Jul 24 23:30:16.475679 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.474945    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.477488 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.477225    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8664ff4bc4-5ffwz"]
Jul 24 23:30:16.485823 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.485795    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 24 23:30:16.623372 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.623329    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-service-ca\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.623587 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.623439    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-config\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.623587 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.623514    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-trusted-ca-bundle\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.623587 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.623535    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9lh7\" (UniqueName: \"kubernetes.io/projected/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-kube-api-access-s9lh7\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.623587 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.623560    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-oauth-config\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.623806 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.623635    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-oauth-serving-cert\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.623806 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.623661    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-serving-cert\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.724220    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-config\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.724269    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-trusted-ca-bundle\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.724296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9lh7\" (UniqueName: \"kubernetes.io/projected/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-kube-api-access-s9lh7\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.724324    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-oauth-config\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.724368    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-oauth-serving-cert\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.724398    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-serving-cert\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.724495    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-service-ca\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.725131    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-config\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.725321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.725233    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-service-ca\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.727415 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.726732    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-trusted-ca-bundle\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.727415 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.727357    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-oauth-serving-cert\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.729815 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.729791    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-oauth-config\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.737288 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.737172    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9lh7\" (UniqueName: \"kubernetes.io/projected/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-kube-api-access-s9lh7\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.737875 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.737817    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-serving-cert\") pod \"console-8664ff4bc4-5ffwz\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") " pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:16.796270 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:16.796016    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:22.216112 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.214804    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8664ff4bc4-5ffwz"]
Jul 24 23:30:22.225351 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:22.225294    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc99d534_b0cc_4ba5_8a88_6c1bd808aa95.slice/crio-d43fec1585b8c5fb7859ece53f513846e45047083c70f87bb84f7bbc46518448 WatchSource:0}: Error finding container d43fec1585b8c5fb7859ece53f513846e45047083c70f87bb84f7bbc46518448: Status 404 returned error can't find the container with id d43fec1585b8c5fb7859ece53f513846e45047083c70f87bb84f7bbc46518448
Jul 24 23:30:22.481018 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.480978    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c755bfbd-sp4r9" event={"ID":"fbb6ab74-a15a-4830-af34-6ddf853c57df","Type":"ContainerStarted","Data":"4af5863604944cc715afea7447180aea47f0c83f1d312f0e63ab1ba0d6fe7487"}
Jul 24 23:30:22.482379 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.482355    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84pqs" event={"ID":"648fb8e3-226a-422a-8e0b-cbdf0e58a310","Type":"ContainerStarted","Data":"5477780db3f861d599cfd06a734185c6d767203eece33734acc68198dbd4a05d"}
Jul 24 23:30:22.483857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.483797    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fg5dg" event={"ID":"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f","Type":"ContainerStarted","Data":"c6d6fcb0af1b33aaf9a3cc6fd7dec503ad08b6da1c7060dda9f667fac922f342"}
Jul 24 23:30:22.485622 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.485593    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dhmkj" event={"ID":"e929bc11-c049-490a-a60e-48be763e1b27","Type":"ContainerStarted","Data":"df759a4a7ca2c108f2d58fb8d031e78e5ad680a83fe61a0dd9916e612cba0e45"}
Jul 24 23:30:22.485730 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.485633    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dhmkj" event={"ID":"e929bc11-c049-490a-a60e-48be763e1b27","Type":"ContainerStarted","Data":"be526af9522e0a53acec4a2432fe0b8c87e7803a26a431e6d2dd0d3e5fc6789d"}
Jul 24 23:30:22.485730 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.485690    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:22.487023 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.487004    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664ff4bc4-5ffwz" event={"ID":"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95","Type":"ContainerStarted","Data":"c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2"}
Jul 24 23:30:22.487109 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.487036    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664ff4bc4-5ffwz" event={"ID":"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95","Type":"ContainerStarted","Data":"d43fec1585b8c5fb7859ece53f513846e45047083c70f87bb84f7bbc46518448"}
Jul 24 23:30:22.497605 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.497564    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64c755bfbd-sp4r9" podStartSLOduration=1.022042689 podStartE2EDuration="7.497548789s" totalImagesPullingTime="6.4755061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:22.496775418 +0000 UTC m=+56.992487293" watchObservedRunningTime="2026-07-24 23:30:22.497548789 +0000 UTC m=+56.993260664"
Jul 24 23:30:22.512746 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.512685    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-84pqs" podStartSLOduration=1.882075282 podStartE2EDuration="9.512668905s" totalImagesPullingTime="7.630593623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:22.511964121 +0000 UTC m=+57.007675997" watchObservedRunningTime="2026-07-24 23:30:22.512668905 +0000 UTC m=+57.008380758"
Jul 24 23:30:22.533560 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.533505    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dhmkj" podStartSLOduration=1.91163909 podStartE2EDuration="9.533482754s" totalImagesPullingTime="7.621843664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:22.533441195 +0000 UTC m=+57.029153070" watchObservedRunningTime="2026-07-24 23:30:22.533482754 +0000 UTC m=+57.029194629"
Jul 24 23:30:22.549361 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:22.549257    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8664ff4bc4-5ffwz" podStartSLOduration=6.549236288 podStartE2EDuration="6.549236288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:22.547937907 +0000 UTC m=+57.043649783" watchObservedRunningTime="2026-07-24 23:30:22.549236288 +0000 UTC m=+57.044948166"
Jul 24 23:30:25.370112 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:25.370072    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:25.370630 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:25.370153    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:25.371849 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:25.371801    2572 patch_prober.go:28] interesting pod/console-64c755bfbd-sp4r9 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 24 23:30:25.371964 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:25.371869    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-64c755bfbd-sp4r9" podUID="fbb6ab74-a15a-4830-af34-6ddf853c57df" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 24 23:30:25.396161 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:25.396126    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7qjmg"
Jul 24 23:30:26.505872 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:26.505830    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fg5dg" event={"ID":"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f","Type":"ContainerStarted","Data":"f1b6a9d84caadb027e311ad391d4bf2e6f8d47619614fe3d86b25dc1b7fc7792"}
Jul 24 23:30:26.796674 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:26.796573    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:26.796674 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:26.796649    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:26.798199 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:26.798169    2572 patch_prober.go:28] interesting pod/console-8664ff4bc4-5ffwz container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" start-of-body=
Jul 24 23:30:26.798332 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:26.798219    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8664ff4bc4-5ffwz" podUID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" containerName="console" probeResult="failure" output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused"
Jul 24 23:30:27.563399 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.563352    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"]
Jul 24 23:30:27.566590 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.566567    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.568888 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.568862    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 23:30:27.569036 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.568947    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dk9sd\""
Jul 24 23:30:27.569036 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.568958    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 23:30:27.569036 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.568996    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 23:30:27.569036 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.568958    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 24 23:30:27.569264 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.569001    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 24 23:30:27.577225 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.577183    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"]
Jul 24 23:30:27.605838 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.605802    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7q8td"]
Jul 24 23:30:27.608888 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.608860    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.611401 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.611374    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 23:30:27.611401 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.611385    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 23:30:27.611616 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.611385    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 23:30:27.611885 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.611858    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n4stc\""
Jul 24 23:30:27.615142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.615112    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.615278 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.615155    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g26t\" (UniqueName: \"kubernetes.io/projected/c229b15b-5001-412f-9027-62e76f2396e1-kube-api-access-5g26t\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.615278 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.615260    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.615397 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.615330    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c229b15b-5001-412f-9027-62e76f2396e1-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.716754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.716714    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.716939 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.716761    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g26t\" (UniqueName: \"kubernetes.io/projected/c229b15b-5001-412f-9027-62e76f2396e1-kube-api-access-5g26t\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.716939 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.716791    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-wtmp\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.716939 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:27.716848    2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 24 23:30:27.716939 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.716857    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.716939 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.716886    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.716939 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:27.716925    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls podName:c229b15b-5001-412f-9027-62e76f2396e1 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:28.216904804 +0000 UTC m=+62.712616660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-jbfbn" (UID: "c229b15b-5001-412f-9027-62e76f2396e1") : secret "openshift-state-metrics-tls" not found
Jul 24 23:30:27.717309 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717059    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-root\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.717309 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717087    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba37a67a-9c72-485c-9f82-5de223d85fcb-metrics-client-ca\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.717309 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717115    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-tls\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.717309 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717149    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c229b15b-5001-412f-9027-62e76f2396e1-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.717309 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717178    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c884f\" (UniqueName: \"kubernetes.io/projected/ba37a67a-9c72-485c-9f82-5de223d85fcb-kube-api-access-c884f\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.717309 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717259    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-sys\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.717584 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717319    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-textfile\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.717584 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717402    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.717939 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.717913    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c229b15b-5001-412f-9027-62e76f2396e1-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.719733 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.719712    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.724876 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.724845    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g26t\" (UniqueName: \"kubernetes.io/projected/c229b15b-5001-412f-9027-62e76f2396e1-kube-api-access-5g26t\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:27.818094 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818012    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c884f\" (UniqueName: \"kubernetes.io/projected/ba37a67a-9c72-485c-9f82-5de223d85fcb-kube-api-access-c884f\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818094 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818074    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-sys\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818307 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818099    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-textfile\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818307 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818133    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818307 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818205    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-wtmp\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818307 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818260    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818307 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818290    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-root\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818310    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba37a67a-9c72-485c-9f82-5de223d85fcb-metrics-client-ca\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818333    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-tls\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818373    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-wtmp\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818202    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-sys\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818500 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:27.818436    2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 23:30:27.818500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818444    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba37a67a-9c72-485c-9f82-5de223d85fcb-root\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818476    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-textfile\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.818949 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:27.818518    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-tls podName:ba37a67a-9c72-485c-9f82-5de223d85fcb nodeName:}" failed. No retries permitted until 2026-07-24 23:30:28.318499465 +0000 UTC m=+62.814211326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-tls") pod "node-exporter-7q8td" (UID: "ba37a67a-9c72-485c-9f82-5de223d85fcb") : secret "node-exporter-tls" not found
Jul 24 23:30:27.818949 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818804    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.819080 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.818947    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba37a67a-9c72-485c-9f82-5de223d85fcb-metrics-client-ca\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.821194 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.821088    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:27.826253 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:27.826223    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c884f\" (UniqueName: \"kubernetes.io/projected/ba37a67a-9c72-485c-9f82-5de223d85fcb-kube-api-access-c884f\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:28.221832 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:28.221706    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:28.221990 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:28.221889    2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 24 23:30:28.221990 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:28.221966    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls podName:c229b15b-5001-412f-9027-62e76f2396e1 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:29.221950356 +0000 UTC m=+63.717662212 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-jbfbn" (UID: "c229b15b-5001-412f-9027-62e76f2396e1") : secret "openshift-state-metrics-tls" not found
Jul 24 23:30:28.322496 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:28.322448    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-tls\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:28.325768 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:28.325739    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba37a67a-9c72-485c-9f82-5de223d85fcb-node-exporter-tls\") pod \"node-exporter-7q8td\" (UID: \"ba37a67a-9c72-485c-9f82-5de223d85fcb\") " pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:28.519087 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:28.519028    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7q8td"
Jul 24 23:30:29.097721 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:29.097680    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba37a67a_9c72_485c_9f82_5de223d85fcb.slice/crio-446cda983a85bcb11f92ff92c58ed434e7d1438243ed4eb4aeccc3dce2807e91 WatchSource:0}: Error finding container 446cda983a85bcb11f92ff92c58ed434e7d1438243ed4eb4aeccc3dce2807e91: Status 404 returned error can't find the container with id 446cda983a85bcb11f92ff92c58ed434e7d1438243ed4eb4aeccc3dce2807e91
Jul 24 23:30:29.231860 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.231814    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:29.234230 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.234210    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c229b15b-5001-412f-9027-62e76f2396e1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-jbfbn\" (UID: \"c229b15b-5001-412f-9027-62e76f2396e1\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:29.377086 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.376981    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"
Jul 24 23:30:29.512278 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.512245    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn"]
Jul 24 23:30:29.516321 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:29.516290    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc229b15b_5001_412f_9027_62e76f2396e1.slice/crio-0b1fe54d8ac1b4f604cc843cbea41b4288220f33600d99b16c5a5256c1e4aba4 WatchSource:0}: Error finding container 0b1fe54d8ac1b4f604cc843cbea41b4288220f33600d99b16c5a5256c1e4aba4: Status 404 returned error can't find the container with id 0b1fe54d8ac1b4f604cc843cbea41b4288220f33600d99b16c5a5256c1e4aba4
Jul 24 23:30:29.523144 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.523064    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7q8td" event={"ID":"ba37a67a-9c72-485c-9f82-5de223d85fcb","Type":"ContainerStarted","Data":"446cda983a85bcb11f92ff92c58ed434e7d1438243ed4eb4aeccc3dce2807e91"}
Jul 24 23:30:29.524520 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.524486    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn" event={"ID":"c229b15b-5001-412f-9027-62e76f2396e1","Type":"ContainerStarted","Data":"0b1fe54d8ac1b4f604cc843cbea41b4288220f33600d99b16c5a5256c1e4aba4"}
Jul 24 23:30:29.527016 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.526982    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fg5dg" event={"ID":"46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f","Type":"ContainerStarted","Data":"d50da52f8c96970f610808ca4b072f2cfd69f0aa60ea96a6f80d277f5b7490c0"}
Jul 24 23:30:29.541785 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.541750    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"]
Jul 24 23:30:29.546663 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.546635    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.549263 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.549228    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 23:30:29.549416 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.549286    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 23:30:29.549416 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.549342    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 23:30:29.549416 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.549230    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9m895\""
Jul 24 23:30:29.549416 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.549240    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-blksjevolo8ij\""
Jul 24 23:30:29.549636 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.549235    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 23:30:29.550343 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.550321    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 23:30:29.550708 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.550662    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fg5dg" podStartSLOduration=2.054240268 podStartE2EDuration="16.550645721s" totalImagesPullingTime="14.496405453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:30:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:29.550199992 +0000 UTC m=+64.045911871" watchObservedRunningTime="2026-07-24 23:30:29.550645721 +0000 UTC m=+64.046357596"
Jul 24 23:30:29.558417 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.558387    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"]
Jul 24 23:30:29.635574 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.635537    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.635727 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.635598    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-grpc-tls\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.635784 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.635751    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba4e419-9c48-496d-a88b-154683c1338a-metrics-client-ca\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.636013 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.635854    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.636147 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.636028    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.636147 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.636105    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-tls\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.636147 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.636134    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttrxn\" (UniqueName: \"kubernetes.io/projected/dba4e419-9c48-496d-a88b-154683c1338a-kube-api-access-ttrxn\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.636296 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.636205    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.737601 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.737556    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba4e419-9c48-496d-a88b-154683c1338a-metrics-client-ca\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.737743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.737613    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.737743 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.737642    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.737834 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.737805    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-tls\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.737905 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.737847    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttrxn\" (UniqueName: \"kubernetes.io/projected/dba4e419-9c48-496d-a88b-154683c1338a-kube-api-access-ttrxn\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.737905 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.737893    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.738648 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.738140    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.738648 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.738196    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-grpc-tls\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.738648 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.738579    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dba4e419-9c48-496d-a88b-154683c1338a-metrics-client-ca\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.740910 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.740881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.740910 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.740908    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.741111 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.740936    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.741205 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.741185    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-grpc-tls\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.741328 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.741308    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-tls\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.741421 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.741396    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dba4e419-9c48-496d-a88b-154683c1338a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.745782 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.745754    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttrxn\" (UniqueName: \"kubernetes.io/projected/dba4e419-9c48-496d-a88b-154683c1338a-kube-api-access-ttrxn\") pod \"thanos-querier-7dfbbf9f65-s75k4\" (UID: \"dba4e419-9c48-496d-a88b-154683c1338a\") " pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:29.880110 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:29.880013    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:30.061210 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:30.061169    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba4e419_9c48_496d_a88b_154683c1338a.slice/crio-84c0c0509c38720b7065c320999f658ce0ef34defe08c7c0f9b1865744b50a81 WatchSource:0}: Error finding container 84c0c0509c38720b7065c320999f658ce0ef34defe08c7c0f9b1865744b50a81: Status 404 returned error can't find the container with id 84c0c0509c38720b7065c320999f658ce0ef34defe08c7c0f9b1865744b50a81
Jul 24 23:30:30.072900 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.072868    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"]
Jul 24 23:30:30.532952 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.532909    2572 generic.go:356] "Generic (PLEG): container finished" podID="ba37a67a-9c72-485c-9f82-5de223d85fcb" containerID="115d8f5120016e922b4ffeb8936283aad6d1e46ba2652926b760c6ef70e10b55" exitCode=0
Jul 24 23:30:30.533433 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.533036    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7q8td" event={"ID":"ba37a67a-9c72-485c-9f82-5de223d85fcb","Type":"ContainerDied","Data":"115d8f5120016e922b4ffeb8936283aad6d1e46ba2652926b760c6ef70e10b55"}
Jul 24 23:30:30.535495 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.535463    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn" event={"ID":"c229b15b-5001-412f-9027-62e76f2396e1","Type":"ContainerStarted","Data":"16292188bdf82dd999aab93c2266c0353c78ead7c6baf85d1f975ae5cb834b8b"}
Jul 24 23:30:30.535612 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.535520    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn" event={"ID":"c229b15b-5001-412f-9027-62e76f2396e1","Type":"ContainerStarted","Data":"3aa321bc88b478c48582b67a77ae53fb086a1df8840037ccf4fe2176570ea9b1"}
Jul 24 23:30:30.536852 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.536828    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" event={"ID":"dba4e419-9c48-496d-a88b-154683c1338a","Type":"ContainerStarted","Data":"84c0c0509c38720b7065c320999f658ce0ef34defe08c7c0f9b1865744b50a81"}
Jul 24 23:30:30.748283 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.748230    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:30.748508 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.748313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:30.750373 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.750334    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 23:30:30.750506 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.750422    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 23:30:30.762180 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.762148    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6634d566-76cf-4c96-9668-bc6ee975ed83-original-pull-secret\") pod \"global-pull-secret-syncer-n6fg5\" (UID: \"6634d566-76cf-4c96-9668-bc6ee975ed83\") " pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:30.762510 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.762487    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24645aa9-1df4-4033-84e5-00059d19bf6d-metrics-certs\") pod \"network-metrics-daemon-wvlgk\" (UID: \"24645aa9-1df4-4033-84e5-00059d19bf6d\") " pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:30.849484 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.849372    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:30.851863 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.851835    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 23:30:30.862200 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.862165    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 23:30:30.874033 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.873996    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tvxk\" (UniqueName: \"kubernetes.io/projected/4cd16aae-7f7b-4636-bd97-bec54f284e0f-kube-api-access-5tvxk\") pod \"network-check-target-cqzrz\" (UID: \"4cd16aae-7f7b-4636-bd97-bec54f284e0f\") " pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:30.879589 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.879555    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzdt8\""
Jul 24 23:30:30.886445 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.886409    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6fg5"
Jul 24 23:30:30.888589 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.888565    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:30.896989 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.896951    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t27k5\""
Jul 24 23:30:30.904923 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:30.904887    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wvlgk"
Jul 24 23:30:31.064493 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.064436    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cqzrz"]
Jul 24 23:30:31.072009 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:31.071973    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd16aae_7f7b_4636_bd97_bec54f284e0f.slice/crio-b89d43970f2ef07507fd1c34212dc1e98a90e4f509f4ff7eaa6b5a6a5769798d WatchSource:0}: Error finding container b89d43970f2ef07507fd1c34212dc1e98a90e4f509f4ff7eaa6b5a6a5769798d: Status 404 returned error can't find the container with id b89d43970f2ef07507fd1c34212dc1e98a90e4f509f4ff7eaa6b5a6a5769798d
Jul 24 23:30:31.082471 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.082419    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n6fg5"]
Jul 24 23:30:31.086802 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:31.086772    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6634d566_76cf_4c96_9668_bc6ee975ed83.slice/crio-36208fc2039d507f2667d08056a5aab2c9d3ee8a9a455e2687ed50e885d73b2d WatchSource:0}: Error finding container 36208fc2039d507f2667d08056a5aab2c9d3ee8a9a455e2687ed50e885d73b2d: Status 404 returned error can't find the container with id 36208fc2039d507f2667d08056a5aab2c9d3ee8a9a455e2687ed50e885d73b2d
Jul 24 23:30:31.108467 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.108309    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wvlgk"]
Jul 24 23:30:31.111678 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:31.111642    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24645aa9_1df4_4033_84e5_00059d19bf6d.slice/crio-6a15fde9d5a1a5a8309d7a97300ced603fc065d59129a202c076c7ad0f5ddc64 WatchSource:0}: Error finding container 6a15fde9d5a1a5a8309d7a97300ced603fc065d59129a202c076c7ad0f5ddc64: Status 404 returned error can't find the container with id 6a15fde9d5a1a5a8309d7a97300ced603fc065d59129a202c076c7ad0f5ddc64
Jul 24 23:30:31.546821 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.546711    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7q8td" event={"ID":"ba37a67a-9c72-485c-9f82-5de223d85fcb","Type":"ContainerStarted","Data":"1b045859199743f44559aef7dcd1ddf7cb849b15c0d2c26d02606342c5e54111"}
Jul 24 23:30:31.546821 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.546779    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7q8td" event={"ID":"ba37a67a-9c72-485c-9f82-5de223d85fcb","Type":"ContainerStarted","Data":"46170e4f4df185ec586b97c0001724202a98bfb876892fd595565ef06989f0c2"}
Jul 24 23:30:31.548975 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.548933    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wvlgk" event={"ID":"24645aa9-1df4-4033-84e5-00059d19bf6d","Type":"ContainerStarted","Data":"6a15fde9d5a1a5a8309d7a97300ced603fc065d59129a202c076c7ad0f5ddc64"}
Jul 24 23:30:31.551180 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.551145    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n6fg5" event={"ID":"6634d566-76cf-4c96-9668-bc6ee975ed83","Type":"ContainerStarted","Data":"36208fc2039d507f2667d08056a5aab2c9d3ee8a9a455e2687ed50e885d73b2d"}
Jul 24 23:30:31.555111 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.554844    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn" event={"ID":"c229b15b-5001-412f-9027-62e76f2396e1","Type":"ContainerStarted","Data":"d93cd6a2c8161fb7df8084320cc839897ecbe34fd1708a8bf1af8d6ceb25eb4a"}
Jul 24 23:30:31.556485 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.556441    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cqzrz" event={"ID":"4cd16aae-7f7b-4636-bd97-bec54f284e0f","Type":"ContainerStarted","Data":"b89d43970f2ef07507fd1c34212dc1e98a90e4f509f4ff7eaa6b5a6a5769798d"}
Jul 24 23:30:31.565932 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.565585    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7q8td" podStartSLOduration=3.623504898 podStartE2EDuration="4.565562479s" totalImagesPullingTime="873.592372ms" totalInitContainerRuntime="68.465209ms" isStatefulPod=true podCreationTimestamp="2026-07-24 23:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:31.563966461 +0000 UTC m=+66.059678343" watchObservedRunningTime="2026-07-24 23:30:31.565562479 +0000 UTC m=+66.061274355"
Jul 24 23:30:31.582764 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.582692    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-jbfbn" podStartSLOduration=3.601844598 podStartE2EDuration="4.582667011s" totalImagesPullingTime="980.822413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:31.581776176 +0000 UTC m=+66.077488053" watchObservedRunningTime="2026-07-24 23:30:31.582667011 +0000 UTC m=+66.078378885"
Jul 24 23:30:31.781067 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.781013    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-794cb85898-g7wmw"]
Jul 24 23:30:31.785579 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.785503    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.788269 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.788245    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-if5ildbvhmj8\""
Jul 24 23:30:31.788861 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.788446    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 23:30:31.788861 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.788543    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 24 23:30:31.788861 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.788621    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 24 23:30:31.788861 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.788651    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7tz9m\""
Jul 24 23:30:31.788861 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.788762    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 24 23:30:31.793545 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.793441    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-794cb85898-g7wmw"]
Jul 24 23:30:31.858887 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.858787    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-metrics-server-audit-profiles\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.858887 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.858853    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-audit-log\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.859113 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.858941    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-client-ca-bundle\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.859113 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.858983    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-secret-metrics-server-client-certs\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.859113 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.859010    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r99ld\" (UniqueName: \"kubernetes.io/projected/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-kube-api-access-r99ld\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.859113 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.859048    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-secret-metrics-server-tls\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.859113 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.859089    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.960380 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.960343    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-audit-log\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.960649 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.960405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-client-ca-bundle\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.960649 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.960440    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-secret-metrics-server-client-certs\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.960783 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.960695    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r99ld\" (UniqueName: \"kubernetes.io/projected/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-kube-api-access-r99ld\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.960783 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.960773    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-secret-metrics-server-tls\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.960894 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.960823    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.960947 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.960914    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-metrics-server-audit-profiles\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.962080 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.961353    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-audit-log\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.962080 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.962007    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.962312 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.962149    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-metrics-server-audit-profiles\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.963791 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.963744    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-client-ca-bundle\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.963956 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.963786    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-secret-metrics-server-tls\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.964293 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.964269    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-secret-metrics-server-client-certs\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:31.970328 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:31.970270    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r99ld\" (UniqueName: \"kubernetes.io/projected/afb7ed68-2a6a-43c3-87bb-4a6bffb48544-kube-api-access-r99ld\") pod \"metrics-server-794cb85898-g7wmw\" (UID: \"afb7ed68-2a6a-43c3-87bb-4a6bffb48544\") " pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:32.104051 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.104011    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:32.351997 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.351960    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"]
Jul 24 23:30:32.356219 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.355183    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:32.356911 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.356548    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c755bfbd-sp4r9"]
Jul 24 23:30:32.357673 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.357647    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 23:30:32.357797 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.357713    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cjbmp\""
Jul 24 23:30:32.370781 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.370751    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"]
Jul 24 23:30:32.389301 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.388065    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f58777496-l66xr"]
Jul 24 23:30:32.392041 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.392015    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.403491 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.403412    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f58777496-l66xr"]
Jul 24 23:30:32.466276 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466235    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-service-ca\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.466486 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466293    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-564dw\" (UniqueName: \"kubernetes.io/projected/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-kube-api-access-564dw\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.466486 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466353    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-trusted-ca-bundle\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.466486 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466388    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvhlw\" (UniqueName: \"kubernetes.io/projected/d60abb81-f81d-40d4-be40-47268704771e-kube-api-access-fvhlw\") pod \"monitoring-plugin-68b7fc9574-br9cj\" (UID: \"d60abb81-f81d-40d4-be40-47268704771e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:32.466486 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466426    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d60abb81-f81d-40d4-be40-47268704771e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-br9cj\" (UID: \"d60abb81-f81d-40d4-be40-47268704771e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:32.466486 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466470    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-serving-cert\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.466755 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466498    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-oauth-serving-cert\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.466755 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466543    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-config\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.466755 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.466572    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-oauth-config\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.495689 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.495176    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dhmkj"
Jul 24 23:30:32.567362 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567337    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-config\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567383    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-oauth-config\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567416    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-service-ca\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567494    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-564dw\" (UniqueName: \"kubernetes.io/projected/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-kube-api-access-564dw\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567542    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-trusted-ca-bundle\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567597    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvhlw\" (UniqueName: \"kubernetes.io/projected/d60abb81-f81d-40d4-be40-47268704771e-kube-api-access-fvhlw\") pod \"monitoring-plugin-68b7fc9574-br9cj\" (UID: \"d60abb81-f81d-40d4-be40-47268704771e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d60abb81-f81d-40d4-be40-47268704771e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-br9cj\" (UID: \"d60abb81-f81d-40d4-be40-47268704771e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567689    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-serving-cert\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.567754 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.567715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-oauth-serving-cert\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.568389 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.568212    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-config\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.576283 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.575806    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-service-ca\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.576876 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.576825    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-oauth-serving-cert\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.577375 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:32.577097    2572 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 24 23:30:32.577483 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.577465    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-trusted-ca-bundle\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.581267 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.580991    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-serving-cert\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.582164 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:30:32.582133    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60abb81-f81d-40d4-be40-47268704771e-monitoring-plugin-cert podName:d60abb81-f81d-40d4-be40-47268704771e nodeName:}" failed. No retries permitted until 2026-07-24 23:30:33.082068139 +0000 UTC m=+67.577779995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d60abb81-f81d-40d4-be40-47268704771e-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-br9cj" (UID: "d60abb81-f81d-40d4-be40-47268704771e") : secret "monitoring-plugin-cert" not found
Jul 24 23:30:32.590919 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.589337    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvhlw\" (UniqueName: \"kubernetes.io/projected/d60abb81-f81d-40d4-be40-47268704771e-kube-api-access-fvhlw\") pod \"monitoring-plugin-68b7fc9574-br9cj\" (UID: \"d60abb81-f81d-40d4-be40-47268704771e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:32.590919 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.590733    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-oauth-config\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.591408 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.591383    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-564dw\" (UniqueName: \"kubernetes.io/projected/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-kube-api-access-564dw\") pod \"console-5f58777496-l66xr\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") " pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.706827 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.706626    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:32.739225 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.739100    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-794cb85898-g7wmw"]
Jul 24 23:30:32.779209 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.779179    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"]
Jul 24 23:30:32.790749 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.789676    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.794513 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.792579    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lzqz7\""
Jul 24 23:30:32.794513 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.792639    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 24 23:30:32.794513 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.792889    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 24 23:30:32.794513 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.793064    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 24 23:30:32.794513 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.793091    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 24 23:30:32.804840 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.797678    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 24 23:30:32.804840 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.799125    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"]
Jul 24 23:30:32.804840 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.802790    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 24 23:30:32.871119 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.870987    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ccfv\" (UniqueName: \"kubernetes.io/projected/738bdb77-2ca6-49de-9854-d3b8d6212866-kube-api-access-5ccfv\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.871119 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.871032    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.871119 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.871060    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-telemeter-client-tls\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.871119 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.871087    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-metrics-client-ca\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.871410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.871195    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.871410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.871275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-serving-certs-ca-bundle\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.871410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.871313    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-secret-telemeter-client\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.871410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.871360    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-federate-client-tls\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.897822 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.895892    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f58777496-l66xr"]
Jul 24 23:30:32.906341 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:32.906299    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0d4c81_cfcc_4aa7_8fa7_0191b0c59a19.slice/crio-2c5fb7d899e8483bea25d3adc19252894b2d7f4dce0a2e0b666a3ed49e5a83e4 WatchSource:0}: Error finding container 2c5fb7d899e8483bea25d3adc19252894b2d7f4dce0a2e0b666a3ed49e5a83e4: Status 404 returned error can't find the container with id 2c5fb7d899e8483bea25d3adc19252894b2d7f4dce0a2e0b666a3ed49e5a83e4
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972014    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972086    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-serving-certs-ca-bundle\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972124    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-secret-telemeter-client\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972165    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-federate-client-tls\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972195    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ccfv\" (UniqueName: \"kubernetes.io/projected/738bdb77-2ca6-49de-9854-d3b8d6212866-kube-api-access-5ccfv\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972223    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972251    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-telemeter-client-tls\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.972275    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-metrics-client-ca\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.973395 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.973009    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-metrics-client-ca\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.974134 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.973761    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-serving-certs-ca-bundle\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.974373 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.974346    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738bdb77-2ca6-49de-9854-d3b8d6212866-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.975413 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.975386    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.976216 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.976175    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-secret-telemeter-client\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.976851 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.976572    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-federate-client-tls\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.976851 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.976800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/738bdb77-2ca6-49de-9854-d3b8d6212866-telemeter-client-tls\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:32.982440 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:32.982409    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ccfv\" (UniqueName: \"kubernetes.io/projected/738bdb77-2ca6-49de-9854-d3b8d6212866-kube-api-access-5ccfv\") pod \"telemeter-client-7665fd9b8d-zcrhj\" (UID: \"738bdb77-2ca6-49de-9854-d3b8d6212866\") " pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:33.128440 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.128108    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"
Jul 24 23:30:33.173938 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.173888    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d60abb81-f81d-40d4-be40-47268704771e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-br9cj\" (UID: \"d60abb81-f81d-40d4-be40-47268704771e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:33.177547 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.177503    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d60abb81-f81d-40d4-be40-47268704771e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-br9cj\" (UID: \"d60abb81-f81d-40d4-be40-47268704771e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:33.270138 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.269839    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:33.317298 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.317208    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj"]
Jul 24 23:30:33.322605 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:33.322574    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738bdb77_2ca6_49de_9854_d3b8d6212866.slice/crio-cb02f19c324a8bda72c1da49c954525a9e7a338653089264e6320f4c622bece9 WatchSource:0}: Error finding container cb02f19c324a8bda72c1da49c954525a9e7a338653089264e6320f4c622bece9: Status 404 returned error can't find the container with id cb02f19c324a8bda72c1da49c954525a9e7a338653089264e6320f4c622bece9
Jul 24 23:30:33.458404 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.458366    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"]
Jul 24 23:30:33.462509 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:33.462442    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60abb81_f81d_40d4_be40_47268704771e.slice/crio-2b421feb07f5662597881a585faf48b31a0cf11370704c4254e05afe054d2dce WatchSource:0}: Error finding container 2b421feb07f5662597881a585faf48b31a0cf11370704c4254e05afe054d2dce: Status 404 returned error can't find the container with id 2b421feb07f5662597881a585faf48b31a0cf11370704c4254e05afe054d2dce
Jul 24 23:30:33.570432 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.570395    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" event={"ID":"dba4e419-9c48-496d-a88b-154683c1338a","Type":"ContainerStarted","Data":"a48601af731e26bce807699235adbd561b76bc37333ed50c2712a019f77d4eb6"}
Jul 24 23:30:33.570916 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.570502    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" event={"ID":"dba4e419-9c48-496d-a88b-154683c1338a","Type":"ContainerStarted","Data":"ed2013c94ee6e570ac34354396f9e8629d31584734560df813531f7d663741dc"}
Jul 24 23:30:33.570916 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.570548    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" event={"ID":"dba4e419-9c48-496d-a88b-154683c1338a","Type":"ContainerStarted","Data":"da8d9e67df3150ab1aac2fcca4df3993bf61c30830458e697b128026e6dcb1b4"}
Jul 24 23:30:33.575419 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.574579    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58777496-l66xr" event={"ID":"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19","Type":"ContainerStarted","Data":"1e7ee756847517694ce9258d1bb05e267668bfd5efb34ad852d54ac82e0b3913"}
Jul 24 23:30:33.575419 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.574633    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58777496-l66xr" event={"ID":"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19","Type":"ContainerStarted","Data":"2c5fb7d899e8483bea25d3adc19252894b2d7f4dce0a2e0b666a3ed49e5a83e4"}
Jul 24 23:30:33.579032 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.578962    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw" event={"ID":"afb7ed68-2a6a-43c3-87bb-4a6bffb48544","Type":"ContainerStarted","Data":"96735b81e7ddc54968ed469131a0f42a566f650c7d19f9b99510c689a0fff4b7"}
Jul 24 23:30:33.584811 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.584138    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wvlgk" event={"ID":"24645aa9-1df4-4033-84e5-00059d19bf6d","Type":"ContainerStarted","Data":"06fbb47069f0940fc9bc009f43b54d7f5f342ac1228e96d589a459a871230208"}
Jul 24 23:30:33.584811 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.584191    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wvlgk" event={"ID":"24645aa9-1df4-4033-84e5-00059d19bf6d","Type":"ContainerStarted","Data":"7d91151e9e9bf03d49d6529137b545775e424bf55a44ed747bd0906722def1b3"}
Jul 24 23:30:33.587029 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.586999    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj" event={"ID":"d60abb81-f81d-40d4-be40-47268704771e","Type":"ContainerStarted","Data":"2b421feb07f5662597881a585faf48b31a0cf11370704c4254e05afe054d2dce"}
Jul 24 23:30:33.589333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.589255    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj" event={"ID":"738bdb77-2ca6-49de-9854-d3b8d6212866","Type":"ContainerStarted","Data":"cb02f19c324a8bda72c1da49c954525a9e7a338653089264e6320f4c622bece9"}
Jul 24 23:30:33.620080 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.618568    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wvlgk" podStartSLOduration=66.166472538 podStartE2EDuration="1m7.618544101s" totalImagesPullingTime="1.452071563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:33.616116589 +0000 UTC m=+68.111828471" watchObservedRunningTime="2026-07-24 23:30:33.618544101 +0000 UTC m=+68.114255977"
Jul 24 23:30:33.620080 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.619503    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f58777496-l66xr" podStartSLOduration=1.61948522 podStartE2EDuration="1.61948522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:33.599246992 +0000 UTC m=+68.094958871" watchObservedRunningTime="2026-07-24 23:30:33.61948522 +0000 UTC m=+68.115197076"
Jul 24 23:30:33.793144 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.793104    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 23:30:33.816910 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.816790    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 23:30:33.817096 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.816980    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.820799 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.820079    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 23:30:33.820799 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.820097    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 23:30:33.820799 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.820296    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 23:30:33.820799 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.820368    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 23:30:33.820799 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.820623    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 23:30:33.820799 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.820672    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.822404    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-15nik8ljimgb5\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.823100    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.823579    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.824296    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.824829    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.825094    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.825895    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7pcz2\""
Jul 24 23:30:33.826981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.826557    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 23:30:33.837935 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.837721    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 23:30:33.839480 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.839433    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 23:30:33.881592 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.881372    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.881592 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.881434    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.881592 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.881483    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.881592 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.881515    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.881592 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.881552    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882081    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882123    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882161    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882193    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882221    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882246    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882284    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882316    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882349    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882376    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-config\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2rtj\" (UniqueName: \"kubernetes.io/projected/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-kube-api-access-k2rtj\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882484    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882563    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882600    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.882814 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.882623    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983739 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983704    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983739 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983744    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983765    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983785    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983833    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983869    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983927    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983963    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.983992 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.983992    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984025    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984048    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-config\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984069    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2rtj\" (UniqueName: \"kubernetes.io/projected/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-kube-api-access-k2rtj\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984089    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984168    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984185    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984227    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984237    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.984306 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984251    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.985252 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.984903    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.986345 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.985812    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.986345 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.985867    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.987809 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.987405    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.987809 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.987530    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.987809 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.987770    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.988528 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.988498    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.988708 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.988687    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-config-out\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.989774 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.989236    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.989774 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.989668    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-web-config\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.989774 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.989732    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.989974 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.989824    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.990487 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.990466    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-config\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.991294 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.991251    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.992656 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.992633    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.992737 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.992676    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.993418 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.993399    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.994282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.994257    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.994383 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.994294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:33.996482 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:33.996439    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2rtj\" (UniqueName: \"kubernetes.io/projected/62ad74ea-fb39-47b7-87f9-e8f9374c33aa-kube-api-access-k2rtj\") pod \"prometheus-k8s-0\" (UID: \"62ad74ea-fb39-47b7-87f9-e8f9374c33aa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:34.141520 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:34.141418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:36.803228 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:36.803186    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:36.808001 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:36.807972    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:30:37.645323 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:37.645289    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 23:30:37.867560 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:37.867527    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ad74ea_fb39_47b7_87f9_e8f9374c33aa.slice/crio-942a75225cdbd7e62b564f768f45110d1dbf56726ea4a79df4da360223170b3c WatchSource:0}: Error finding container 942a75225cdbd7e62b564f768f45110d1dbf56726ea4a79df4da360223170b3c: Status 404 returned error can't find the container with id 942a75225cdbd7e62b564f768f45110d1dbf56726ea4a79df4da360223170b3c
Jul 24 23:30:38.618540 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.618499    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cqzrz" event={"ID":"4cd16aae-7f7b-4636-bd97-bec54f284e0f","Type":"ContainerStarted","Data":"f62acd7da6e6273f0e03fdccb5f5dad1ace52d216c38b68ec6fe9ba7e02912c7"}
Jul 24 23:30:38.618729 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.618583    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:30:38.620340 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.620162    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerStarted","Data":"942a75225cdbd7e62b564f768f45110d1dbf56726ea4a79df4da360223170b3c"}
Jul 24 23:30:38.622977 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.622766    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n6fg5" event={"ID":"6634d566-76cf-4c96-9668-bc6ee975ed83","Type":"ContainerStarted","Data":"bc1d793486791486615e1eae18ad48d55f7715ec60b0e95e37c5c289dfdba1a4"}
Jul 24 23:30:38.624974 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.624943    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj" event={"ID":"d60abb81-f81d-40d4-be40-47268704771e","Type":"ContainerStarted","Data":"73d33bee874eeaf4a581d0f003b38a75032bd91fa874ff4a3325b647c71d1457"}
Jul 24 23:30:38.627256 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.626876    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:38.629611 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.629584    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj" event={"ID":"738bdb77-2ca6-49de-9854-d3b8d6212866","Type":"ContainerStarted","Data":"5d7eaaae61c83fcd06be0b9e934b36c5957d7cbd24367cbb1d39078bbce470c6"}
Jul 24 23:30:38.631704 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.631654    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj"
Jul 24 23:30:38.635983 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.634532    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cqzrz" podStartSLOduration=65.814124051 podStartE2EDuration="1m12.634514089s" totalImagesPullingTime="6.820390038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:38.634264333 +0000 UTC m=+73.129976209" watchObservedRunningTime="2026-07-24 23:30:38.634514089 +0000 UTC m=+73.130225964"
Jul 24 23:30:38.635983 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.635249    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" event={"ID":"dba4e419-9c48-496d-a88b-154683c1338a","Type":"ContainerStarted","Data":"0bfd091395dedba83138e4f6ffb78d310078597ee1e7321555d452959dc13a54"}
Jul 24 23:30:38.635983 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.635327    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" event={"ID":"dba4e419-9c48-496d-a88b-154683c1338a","Type":"ContainerStarted","Data":"a572f61fc127df01fb6efbe6b6bf0a3bc325349daafecdf5bde5eb769f0a6a70"}
Jul 24 23:30:38.635983 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.635368    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" event={"ID":"dba4e419-9c48-496d-a88b-154683c1338a","Type":"ContainerStarted","Data":"cd44c559dec26b4bf07d982cb1be4a933afe74c3993e6e732142f562dfdd183b"}
Jul 24 23:30:38.635983 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.635740    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:38.638084 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.638052    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw" event={"ID":"afb7ed68-2a6a-43c3-87bb-4a6bffb48544","Type":"ContainerStarted","Data":"3f6c9d1a27899d71ee780c199dfb29f066175079bffb4f91ca62a7680edbf11e"}
Jul 24 23:30:38.645798 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.645749    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4"
Jul 24 23:30:38.650585 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.650526    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-br9cj" podStartSLOduration=2.215787487 podStartE2EDuration="6.650503104s" totalImagesPullingTime="4.434715617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:38.64920166 +0000 UTC m=+73.144913532" watchObservedRunningTime="2026-07-24 23:30:38.650503104 +0000 UTC m=+73.146214980"
Jul 24 23:30:38.663843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.663770    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n6fg5" podStartSLOduration=65.857758206 podStartE2EDuration="1m12.663746502s" totalImagesPullingTime="6.805988296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:38.662630602 +0000 UTC m=+73.158342476" watchObservedRunningTime="2026-07-24 23:30:38.663746502 +0000 UTC m=+73.159458380"
Jul 24 23:30:38.687102 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.686910    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7dfbbf9f65-s75k4" podStartSLOduration=2.66013666 podStartE2EDuration="9.686893463s" totalImagesPullingTime="7.026756803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:38.686397805 +0000 UTC m=+73.182109674" watchObservedRunningTime="2026-07-24 23:30:38.686893463 +0000 UTC m=+73.182605339"
Jul 24 23:30:38.706304 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:38.706110    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw" podStartSLOduration=2.564055117 podStartE2EDuration="7.70608324s" totalImagesPullingTime="5.142028123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:38.704539359 +0000 UTC m=+73.200251301" watchObservedRunningTime="2026-07-24 23:30:38.70608324 +0000 UTC m=+73.201795118"
Jul 24 23:30:39.643368 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:39.643328    2572 generic.go:356] "Generic (PLEG): container finished" podID="62ad74ea-fb39-47b7-87f9-e8f9374c33aa" containerID="6636464306a6c3f468e1e8bd2e3d766b4f687d62bb17696f2d19a2b6cdf2131c" exitCode=0
Jul 24 23:30:39.643824 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:39.643410    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerDied","Data":"6636464306a6c3f468e1e8bd2e3d766b4f687d62bb17696f2d19a2b6cdf2131c"}
Jul 24 23:30:39.645712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:39.645682    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj" event={"ID":"738bdb77-2ca6-49de-9854-d3b8d6212866","Type":"ContainerStarted","Data":"e7ad5cabf6dd0615f48267111db1d4f62a09ecfe08ad890215076839bf5e1522"}
Jul 24 23:30:39.645788 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:39.645737    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj" event={"ID":"738bdb77-2ca6-49de-9854-d3b8d6212866","Type":"ContainerStarted","Data":"9187188e4609d6b9300d063018e546f4d02e2a384f5ba34115381d3d9aa32ea4"}
Jul 24 23:30:39.693177 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:39.693125    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7665fd9b8d-zcrhj" podStartSLOduration=2.131050247 podStartE2EDuration="7.693105905s" totalImagesPullingTime="5.562055658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:39.691530468 +0000 UTC m=+74.187242343" watchObservedRunningTime="2026-07-24 23:30:39.693105905 +0000 UTC m=+74.188817779"
Jul 24 23:30:40.463951 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.463906    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f58777496-l66xr"]
Jul 24 23:30:40.497728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.497697    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b6cccc4d-wrvz5"]
Jul 24 23:30:40.501072 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.501050    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.510920 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.510879    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b6cccc4d-wrvz5"]
Jul 24 23:30:40.648219 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.648071    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-trusted-ca-bundle\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.648219 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.648122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-service-ca\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.648219 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.648171    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4ztd\" (UniqueName: \"kubernetes.io/projected/0012a204-7a96-45a1-b91b-8b1baa931df5-kube-api-access-r4ztd\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.648736 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.648246    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-serving-cert\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.648736 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.648328    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-console-config\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.648736 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.648415    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-oauth-config\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.648736 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.648494    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-oauth-serving-cert\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.749436 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.749380    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4ztd\" (UniqueName: \"kubernetes.io/projected/0012a204-7a96-45a1-b91b-8b1baa931df5-kube-api-access-r4ztd\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.749640 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.749442    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-serving-cert\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.749640 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.749481    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-console-config\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.749766 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.749738    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-oauth-config\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.749823 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.749813    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-oauth-serving-cert\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.749907 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.749891    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-trusted-ca-bundle\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.750340 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.750314    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-service-ca\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.750771 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.750744    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-oauth-serving-cert\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.750927 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.750906    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-trusted-ca-bundle\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.751159 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.751136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-console-config\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.752068 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.752041    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-service-ca\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.756426 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.753209    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-serving-cert\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.756958 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.756931    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-oauth-config\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.758562 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.758537    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4ztd\" (UniqueName: \"kubernetes.io/projected/0012a204-7a96-45a1-b91b-8b1baa931df5-kube-api-access-r4ztd\") pod \"console-6b6cccc4d-wrvz5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") " pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.812653 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.812602    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:40.962573 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:40.962538    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b6cccc4d-wrvz5"]
Jul 24 23:30:40.966323 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:30:40.966288    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0012a204_7a96_45a1_b91b_8b1baa931df5.slice/crio-3205e526c53b0aa7120c42ae61d7a138049409ff3ea1d136751ba4d51e4a77bf WatchSource:0}: Error finding container 3205e526c53b0aa7120c42ae61d7a138049409ff3ea1d136751ba4d51e4a77bf: Status 404 returned error can't find the container with id 3205e526c53b0aa7120c42ae61d7a138049409ff3ea1d136751ba4d51e4a77bf
Jul 24 23:30:41.657632 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:41.657587    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b6cccc4d-wrvz5" event={"ID":"0012a204-7a96-45a1-b91b-8b1baa931df5","Type":"ContainerStarted","Data":"a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970"}
Jul 24 23:30:41.658084 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:41.657649    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b6cccc4d-wrvz5" event={"ID":"0012a204-7a96-45a1-b91b-8b1baa931df5","Type":"ContainerStarted","Data":"3205e526c53b0aa7120c42ae61d7a138049409ff3ea1d136751ba4d51e4a77bf"}
Jul 24 23:30:41.675043 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:41.674996    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b6cccc4d-wrvz5" podStartSLOduration=1.674980557 podStartE2EDuration="1.674980557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:41.673166139 +0000 UTC m=+76.168878022" watchObservedRunningTime="2026-07-24 23:30:41.674980557 +0000 UTC m=+76.170692432"
Jul 24 23:30:42.663882 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:42.663848    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerStarted","Data":"b079b9a3efeb911cec16717e77658b7fd2179b9e73dced9cac209fc27e0c0fb8"}
Jul 24 23:30:42.708195 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:42.708150    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:30:43.671741 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:43.671707    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerStarted","Data":"1631a501860441c167f54c944121e6e3de0d4c50d1e3fe49c11e0f810a7d40c3"}
Jul 24 23:30:43.672142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:43.671775    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerStarted","Data":"f8955f66fabc27dc2bd90405cbed4c762d3da8d6ef80ff250dd3fba3cc267203"}
Jul 24 23:30:43.672142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:43.671809    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerStarted","Data":"cc5d0c7d82934a4821c02cc807ceed294363ae105d09fbb13587fbb7aa173f5f"}
Jul 24 23:30:43.672142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:43.671852    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerStarted","Data":"9ae322b74d1ead345b4b495d12aff03d51aad7fd5ef574fc47ce29000e6a9504"}
Jul 24 23:30:43.672142 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:43.671877    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"62ad74ea-fb39-47b7-87f9-e8f9374c33aa","Type":"ContainerStarted","Data":"cab6515cc6bdc6f92e3ae9fc68c3ab4f18f0fdb032261006ce3c9835dc9e4171"}
Jul 24 23:30:43.698978 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:43.698892    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.690206433 podStartE2EDuration="10.698867828s" totalImagesPullingTime="3.981279557s" totalInitContainerRuntime="27.381838ms" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:43.698023454 +0000 UTC m=+78.193735330" watchObservedRunningTime="2026-07-24 23:30:43.698867828 +0000 UTC m=+78.194579705"
Jul 24 23:30:44.142162 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:44.142104    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:30:50.813794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:50.813638    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:50.813794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:50.813702    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:50.818526 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:50.818502    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:51.708489 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:51.708447    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:30:51.759675 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:51.759642    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8664ff4bc4-5ffwz"]
Jul 24 23:30:52.104773 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:52.104733    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:52.105151 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:52.104800    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:30:57.392528 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:57.392479    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64c755bfbd-sp4r9" podUID="fbb6ab74-a15a-4830-af34-6ddf853c57df" containerName="console" containerID="cri-o://4af5863604944cc715afea7447180aea47f0c83f1d312f0e63ab1ba0d6fe7487" gracePeriod=15
Jul 24 23:30:57.733388 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:57.733304    2572 generic.go:356] "Generic (PLEG): container finished" podID="fbb6ab74-a15a-4830-af34-6ddf853c57df" containerID="4af5863604944cc715afea7447180aea47f0c83f1d312f0e63ab1ba0d6fe7487" exitCode=0
Jul 24 23:30:57.733388 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:57.733343    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c755bfbd-sp4r9" event={"ID":"fbb6ab74-a15a-4830-af34-6ddf853c57df","Type":"ContainerDied","Data":"4af5863604944cc715afea7447180aea47f0c83f1d312f0e63ab1ba0d6fe7487"}
Jul 24 23:30:58.218322 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.218287    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:58.316427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.316332    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-oauth-config\") pod \"fbb6ab74-a15a-4830-af34-6ddf853c57df\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") "
Jul 24 23:30:58.316427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.316371    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-oauth-serving-cert\") pod \"fbb6ab74-a15a-4830-af34-6ddf853c57df\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") "
Jul 24 23:30:58.316427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.316399    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-serving-cert\") pod \"fbb6ab74-a15a-4830-af34-6ddf853c57df\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") "
Jul 24 23:30:58.316427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.316423    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-config\") pod \"fbb6ab74-a15a-4830-af34-6ddf853c57df\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") "
Jul 24 23:30:58.316798 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.316445    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cchcj\" (UniqueName: \"kubernetes.io/projected/fbb6ab74-a15a-4830-af34-6ddf853c57df-kube-api-access-cchcj\") pod \"fbb6ab74-a15a-4830-af34-6ddf853c57df\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") "
Jul 24 23:30:58.316852 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.316820    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fbb6ab74-a15a-4830-af34-6ddf853c57df" (UID: "fbb6ab74-a15a-4830-af34-6ddf853c57df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:30:58.316852 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.316845    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-config" (OuterVolumeSpecName: "console-config") pod "fbb6ab74-a15a-4830-af34-6ddf853c57df" (UID: "fbb6ab74-a15a-4830-af34-6ddf853c57df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:30:58.318843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.318808    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fbb6ab74-a15a-4830-af34-6ddf853c57df" (UID: "fbb6ab74-a15a-4830-af34-6ddf853c57df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:30:58.318954 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.318890    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbb6ab74-a15a-4830-af34-6ddf853c57df-kube-api-access-cchcj" (OuterVolumeSpecName: "kube-api-access-cchcj") pod "fbb6ab74-a15a-4830-af34-6ddf853c57df" (UID: "fbb6ab74-a15a-4830-af34-6ddf853c57df"). InnerVolumeSpecName "kube-api-access-cchcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:30:58.318954 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.318909    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fbb6ab74-a15a-4830-af34-6ddf853c57df" (UID: "fbb6ab74-a15a-4830-af34-6ddf853c57df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:30:58.417544 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.417511    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-service-ca\") pod \"fbb6ab74-a15a-4830-af34-6ddf853c57df\" (UID: \"fbb6ab74-a15a-4830-af34-6ddf853c57df\") "
Jul 24 23:30:58.417921 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.417739    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-oauth-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:30:58.417921 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.417750    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-oauth-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:30:58.417921 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.417760    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:30:58.417921 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.417769    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-console-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:30:58.417921 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.417779    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cchcj\" (UniqueName: \"kubernetes.io/projected/fbb6ab74-a15a-4830-af34-6ddf853c57df-kube-api-access-cchcj\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:30:58.417921 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.417852    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-service-ca" (OuterVolumeSpecName: "service-ca") pod "fbb6ab74-a15a-4830-af34-6ddf853c57df" (UID: "fbb6ab74-a15a-4830-af34-6ddf853c57df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:30:58.519017 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.518979    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb6ab74-a15a-4830-af34-6ddf853c57df-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:30:58.738500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.738392    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c755bfbd-sp4r9" event={"ID":"fbb6ab74-a15a-4830-af34-6ddf853c57df","Type":"ContainerDied","Data":"5a70df1da662619bf9af83f0e1005041c2819ef82e64537b2e7668fd5aca1e6f"}
Jul 24 23:30:58.738500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.738429    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c755bfbd-sp4r9"
Jul 24 23:30:58.738500 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.738479    2572 scope.go:117] "RemoveContainer" containerID="4af5863604944cc715afea7447180aea47f0c83f1d312f0e63ab1ba0d6fe7487"
Jul 24 23:30:58.758089 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.758056    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c755bfbd-sp4r9"]
Jul 24 23:30:58.760224 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:58.760194    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64c755bfbd-sp4r9"]
Jul 24 23:30:59.965575 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:30:59.965543    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbb6ab74-a15a-4830-af34-6ddf853c57df" path="/var/lib/kubelet/pods/fbb6ab74-a15a-4830-af34-6ddf853c57df/volumes"
Jul 24 23:31:05.488049 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.488003    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f58777496-l66xr" podUID="bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" containerName="console" containerID="cri-o://1e7ee756847517694ce9258d1bb05e267668bfd5efb34ad852d54ac82e0b3913" gracePeriod=15
Jul 24 23:31:05.780380 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.780194    2572 generic.go:356] "Generic (PLEG): container finished" podID="bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" containerID="1e7ee756847517694ce9258d1bb05e267668bfd5efb34ad852d54ac82e0b3913" exitCode=0
Jul 24 23:31:05.780380 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.780274    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58777496-l66xr" event={"ID":"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19","Type":"ContainerDied","Data":"1e7ee756847517694ce9258d1bb05e267668bfd5efb34ad852d54ac82e0b3913"}
Jul 24 23:31:05.780380 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.780328    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58777496-l66xr" event={"ID":"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19","Type":"ContainerDied","Data":"2c5fb7d899e8483bea25d3adc19252894b2d7f4dce0a2e0b666a3ed49e5a83e4"}
Jul 24 23:31:05.780380 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.780349    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c5fb7d899e8483bea25d3adc19252894b2d7f4dce0a2e0b666a3ed49e5a83e4"
Jul 24 23:31:05.792941 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.792913    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:31:05.890820 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.890777    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-trusted-ca-bundle\") pod \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") "
Jul 24 23:31:05.891113 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.891093    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-service-ca\") pod \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") "
Jul 24 23:31:05.891274 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.891258    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-serving-cert\") pod \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") "
Jul 24 23:31:05.891409 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.891395    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-564dw\" (UniqueName: \"kubernetes.io/projected/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-kube-api-access-564dw\") pod \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") "
Jul 24 23:31:05.891604 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.891588    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-oauth-config\") pod \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") "
Jul 24 23:31:05.891748 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.891727    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-config\") pod \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") "
Jul 24 23:31:05.891857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.891844    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-oauth-serving-cert\") pod \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\" (UID: \"bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19\") "
Jul 24 23:31:05.892647 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.892618    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" (UID: "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:05.893200 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.893177    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" (UID: "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:05.893694 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.893673    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-service-ca" (OuterVolumeSpecName: "service-ca") pod "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" (UID: "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:05.895192 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.895137    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-config" (OuterVolumeSpecName: "console-config") pod "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" (UID: "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:05.897539 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.897512    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-kube-api-access-564dw" (OuterVolumeSpecName: "kube-api-access-564dw") pod "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" (UID: "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19"). InnerVolumeSpecName "kube-api-access-564dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:31:05.898415 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.898381    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" (UID: "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:31:05.898950 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.898928    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" (UID: "bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:31:05.992865 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.992821    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:05.992865 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.992856    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:05.993077 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.992896    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-564dw\" (UniqueName: \"kubernetes.io/projected/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-kube-api-access-564dw\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:05.993077 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.992909    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-oauth-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:05.993077 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.992921    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-console-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:05.993077 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.992931    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-oauth-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:05.993077 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:05.992942    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19-trusted-ca-bundle\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:06.783616 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:06.783582    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58777496-l66xr"
Jul 24 23:31:06.799100 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:06.799060    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f58777496-l66xr"]
Jul 24 23:31:06.804495 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:06.804465    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f58777496-l66xr"]
Jul 24 23:31:07.964939 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:07.964906    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" path="/var/lib/kubelet/pods/bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19/volumes"
Jul 24 23:31:09.649230 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:09.649199    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cqzrz"
Jul 24 23:31:12.110512 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:12.110480    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:31:12.114717 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:12.114693    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-794cb85898-g7wmw"
Jul 24 23:31:16.780572 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:16.780535    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8664ff4bc4-5ffwz" podUID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" containerName="console" containerID="cri-o://c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2" gracePeriod=15
Jul 24 23:31:16.804557 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:16.804525    2572 patch_prober.go:28] interesting pod/console-8664ff4bc4-5ffwz container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" start-of-body=
Jul 24 23:31:16.804689 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:16.804590    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-8664ff4bc4-5ffwz" podUID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" containerName="console" probeResult="failure" output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused"
Jul 24 23:31:17.018467 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.018427    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:31:17.094541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.094107    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9lh7\" (UniqueName: \"kubernetes.io/projected/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-kube-api-access-s9lh7\") pod \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") "
Jul 24 23:31:17.094541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.094163    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-service-ca\") pod \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") "
Jul 24 23:31:17.094541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.094198    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-trusted-ca-bundle\") pod \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") "
Jul 24 23:31:17.094541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.094224    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-serving-cert\") pod \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") "
Jul 24 23:31:17.094541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.094256    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-config\") pod \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") "
Jul 24 23:31:17.094541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.094285    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-oauth-serving-cert\") pod \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") "
Jul 24 23:31:17.094541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.094319    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-oauth-config\") pod \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\" (UID: \"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95\") "
Jul 24 23:31:17.095735 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.095361    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" (UID: "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:17.096400 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.096371    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-trusted-ca-bundle\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:17.098360 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.098328    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-config" (OuterVolumeSpecName: "console-config") pod "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" (UID: "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:17.099710 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.099495    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" (UID: "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:17.100000 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.099968    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" (UID: "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:17.101584 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.101560    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-kube-api-access-s9lh7" (OuterVolumeSpecName: "kube-api-access-s9lh7") pod "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" (UID: "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95"). InnerVolumeSpecName "kube-api-access-s9lh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:31:17.101712 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.101691    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" (UID: "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:31:17.101893 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.101870    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" (UID: "cc99d534-b0cc-4ba5-8a88-6c1bd808aa95"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:31:17.197048 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.197010    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-oauth-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:17.197048 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.197040    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9lh7\" (UniqueName: \"kubernetes.io/projected/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-kube-api-access-s9lh7\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:17.197048 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.197050    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:17.197048 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.197059    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:17.197400 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.197068    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-console-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:17.197400 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.197077    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95-oauth-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:31:17.831023 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.830986    2572 generic.go:356] "Generic (PLEG): container finished" podID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" containerID="c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2" exitCode=0
Jul 24 23:31:17.831427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.831036    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8664ff4bc4-5ffwz"
Jul 24 23:31:17.831427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.831074    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664ff4bc4-5ffwz" event={"ID":"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95","Type":"ContainerDied","Data":"c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2"}
Jul 24 23:31:17.831427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.831126    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8664ff4bc4-5ffwz" event={"ID":"cc99d534-b0cc-4ba5-8a88-6c1bd808aa95","Type":"ContainerDied","Data":"d43fec1585b8c5fb7859ece53f513846e45047083c70f87bb84f7bbc46518448"}
Jul 24 23:31:17.831427 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.831147    2572 scope.go:117] "RemoveContainer" containerID="c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2"
Jul 24 23:31:17.840742 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.840712    2572 scope.go:117] "RemoveContainer" containerID="c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2"
Jul 24 23:31:17.841041 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:31:17.841015    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2\": container with ID starting with c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2 not found: ID does not exist" containerID="c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2"
Jul 24 23:31:17.841099 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.841056    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2"} err="failed to get container status \"c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2\": rpc error: code = NotFound desc = could not find container \"c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2\": container with ID starting with c4ce2f8ecbbabbcbf3597726a9d637225db59fa76a4a05348fca2bf4e4640dc2 not found: ID does not exist"
Jul 24 23:31:17.850685 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.850651    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8664ff4bc4-5ffwz"]
Jul 24 23:31:17.854114 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.854086    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8664ff4bc4-5ffwz"]
Jul 24 23:31:17.965495 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:17.965448    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" path="/var/lib/kubelet/pods/cc99d534-b0cc-4ba5-8a88-6c1bd808aa95/volumes"
Jul 24 23:31:34.141627 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:34.141588    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:31:34.162646 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:34.162617    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:31:34.912623 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:34.912592    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 23:31:56.125448 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125409    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fb49894b-gtpb2"]
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125725    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbb6ab74-a15a-4830-af34-6ddf853c57df" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125737    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbb6ab74-a15a-4830-af34-6ddf853c57df" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125749    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125755    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125767    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125772    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125820    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbb6ab74-a15a-4830-af34-6ddf853c57df" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125827    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc99d534-b0cc-4ba5-8a88-6c1bd808aa95" containerName="console"
Jul 24 23:31:56.125866 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.125834    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0d4c81-cfcc-4aa7-8fa7-0191b0c59a19" containerName="console"
Jul 24 23:31:56.129220 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.129200    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.137790 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.137765    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fb49894b-gtpb2"]
Jul 24 23:31:56.236700 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.236657    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-serving-cert\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.236700 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.236700    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-service-ca\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.237157 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.236726    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-oauth-serving-cert\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.237157 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.236763    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl7w7\" (UniqueName: \"kubernetes.io/projected/a5ae654c-dc12-40d3-bc44-04d1e55e95db-kube-api-access-cl7w7\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.237157 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.236787    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-trusted-ca-bundle\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.237157 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.236917    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-config\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.237157 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.236999    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-oauth-config\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.337909 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.337867    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-serving-cert\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.337909 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.337906    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-service-ca\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338165 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.337924    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-oauth-serving-cert\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338165 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.337940    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl7w7\" (UniqueName: \"kubernetes.io/projected/a5ae654c-dc12-40d3-bc44-04d1e55e95db-kube-api-access-cl7w7\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338165 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.338081    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-trusted-ca-bundle\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338165 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.338150    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-config\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338370 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.338232    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-oauth-config\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338790 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.338757    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-service-ca\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338968 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.338931    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-trusted-ca-bundle\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338968 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.338875    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-config\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.338968 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.338772    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-oauth-serving-cert\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.340537 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.340518    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-serving-cert\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.340638 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.340570    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-oauth-config\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.345634 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.345608    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl7w7\" (UniqueName: \"kubernetes.io/projected/a5ae654c-dc12-40d3-bc44-04d1e55e95db-kube-api-access-cl7w7\") pod \"console-7fb49894b-gtpb2\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") " pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.439523 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.439430    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:31:56.578653 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.578620    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fb49894b-gtpb2"]
Jul 24 23:31:56.581184 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:31:56.581152    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ae654c_dc12_40d3_bc44_04d1e55e95db.slice/crio-1a528355d98923bf49faca190e9a0d5b7b833f7c52b1d5894148f8092e7b99fe WatchSource:0}: Error finding container 1a528355d98923bf49faca190e9a0d5b7b833f7c52b1d5894148f8092e7b99fe: Status 404 returned error can't find the container with id 1a528355d98923bf49faca190e9a0d5b7b833f7c52b1d5894148f8092e7b99fe
Jul 24 23:31:56.978639 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.978597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb49894b-gtpb2" event={"ID":"a5ae654c-dc12-40d3-bc44-04d1e55e95db","Type":"ContainerStarted","Data":"9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725"}
Jul 24 23:31:56.978822 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.978657    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb49894b-gtpb2" event={"ID":"a5ae654c-dc12-40d3-bc44-04d1e55e95db","Type":"ContainerStarted","Data":"1a528355d98923bf49faca190e9a0d5b7b833f7c52b1d5894148f8092e7b99fe"}
Jul 24 23:31:56.994732 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:31:56.994680    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fb49894b-gtpb2" podStartSLOduration=0.994661731 podStartE2EDuration="994.661731ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:56.99380809 +0000 UTC m=+151.489519967" watchObservedRunningTime="2026-07-24 23:31:56.994661731 +0000 UTC m=+151.490373605"
Jul 24 23:32:06.440361 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:06.440326    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:32:06.440942 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:06.440398    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:32:06.445348 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:06.445324    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:32:07.040777 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:07.040750    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:32:07.081816 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:07.081776    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b6cccc4d-wrvz5"]
Jul 24 23:32:32.101472 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.101412    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b6cccc4d-wrvz5" podUID="0012a204-7a96-45a1-b91b-8b1baa931df5" containerName="console" containerID="cri-o://a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970" gracePeriod=15
Jul 24 23:32:32.351158 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.351131    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:32:32.444590 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.444486    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-oauth-serving-cert\") pod \"0012a204-7a96-45a1-b91b-8b1baa931df5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") "
Jul 24 23:32:32.444590 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.444558    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-serving-cert\") pod \"0012a204-7a96-45a1-b91b-8b1baa931df5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") "
Jul 24 23:32:32.444828 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.444613    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-trusted-ca-bundle\") pod \"0012a204-7a96-45a1-b91b-8b1baa931df5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") "
Jul 24 23:32:32.444828 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.444631    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-service-ca\") pod \"0012a204-7a96-45a1-b91b-8b1baa931df5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") "
Jul 24 23:32:32.444828 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.444647    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-console-config\") pod \"0012a204-7a96-45a1-b91b-8b1baa931df5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") "
Jul 24 23:32:32.444828 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.444669    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-oauth-config\") pod \"0012a204-7a96-45a1-b91b-8b1baa931df5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") "
Jul 24 23:32:32.444828 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.444731    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4ztd\" (UniqueName: \"kubernetes.io/projected/0012a204-7a96-45a1-b91b-8b1baa931df5-kube-api-access-r4ztd\") pod \"0012a204-7a96-45a1-b91b-8b1baa931df5\" (UID: \"0012a204-7a96-45a1-b91b-8b1baa931df5\") "
Jul 24 23:32:32.445075 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.445050    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0012a204-7a96-45a1-b91b-8b1baa931df5" (UID: "0012a204-7a96-45a1-b91b-8b1baa931df5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:32:32.445131 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.445072    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-console-config" (OuterVolumeSpecName: "console-config") pod "0012a204-7a96-45a1-b91b-8b1baa931df5" (UID: "0012a204-7a96-45a1-b91b-8b1baa931df5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:32:32.445131 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.445061    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-service-ca" (OuterVolumeSpecName: "service-ca") pod "0012a204-7a96-45a1-b91b-8b1baa931df5" (UID: "0012a204-7a96-45a1-b91b-8b1baa931df5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:32:32.445131 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.445099    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0012a204-7a96-45a1-b91b-8b1baa931df5" (UID: "0012a204-7a96-45a1-b91b-8b1baa931df5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:32:32.446921 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.446892    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0012a204-7a96-45a1-b91b-8b1baa931df5" (UID: "0012a204-7a96-45a1-b91b-8b1baa931df5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:32:32.447034 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.446949    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0012a204-7a96-45a1-b91b-8b1baa931df5" (UID: "0012a204-7a96-45a1-b91b-8b1baa931df5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:32:32.447034 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.446990    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0012a204-7a96-45a1-b91b-8b1baa931df5-kube-api-access-r4ztd" (OuterVolumeSpecName: "kube-api-access-r4ztd") pod "0012a204-7a96-45a1-b91b-8b1baa931df5" (UID: "0012a204-7a96-45a1-b91b-8b1baa931df5"). InnerVolumeSpecName "kube-api-access-r4ztd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:32:32.546171 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.546133    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4ztd\" (UniqueName: \"kubernetes.io/projected/0012a204-7a96-45a1-b91b-8b1baa931df5-kube-api-access-r4ztd\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:32:32.546171 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.546165    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-oauth-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:32:32.546171 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.546176    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:32:32.546171 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.546187    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-trusted-ca-bundle\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:32:32.546448 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.546210    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:32:32.546448 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.546219    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0012a204-7a96-45a1-b91b-8b1baa931df5-console-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:32:32.546448 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:32.546228    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0012a204-7a96-45a1-b91b-8b1baa931df5-console-oauth-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:32:33.133570 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.133530    2572 generic.go:356] "Generic (PLEG): container finished" podID="0012a204-7a96-45a1-b91b-8b1baa931df5" containerID="a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970" exitCode=0
Jul 24 23:32:33.134054 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.133584    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b6cccc4d-wrvz5"
Jul 24 23:32:33.134054 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.133585    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b6cccc4d-wrvz5" event={"ID":"0012a204-7a96-45a1-b91b-8b1baa931df5","Type":"ContainerDied","Data":"a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970"}
Jul 24 23:32:33.134054 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.133713    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b6cccc4d-wrvz5" event={"ID":"0012a204-7a96-45a1-b91b-8b1baa931df5","Type":"ContainerDied","Data":"3205e526c53b0aa7120c42ae61d7a138049409ff3ea1d136751ba4d51e4a77bf"}
Jul 24 23:32:33.134054 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.133741    2572 scope.go:117] "RemoveContainer" containerID="a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970"
Jul 24 23:32:33.142659 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.142641    2572 scope.go:117] "RemoveContainer" containerID="a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970"
Jul 24 23:32:33.142990 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:32:33.142969    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970\": container with ID starting with a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970 not found: ID does not exist" containerID="a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970"
Jul 24 23:32:33.143028 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.143000    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970"} err="failed to get container status \"a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970\": rpc error: code = NotFound desc = could not find container \"a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970\": container with ID starting with a094bb1d1bdabea11481722c38b7c378d3e15191e3e337f5a75ff8a8b4e3b970 not found: ID does not exist"
Jul 24 23:32:33.154541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.154507    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b6cccc4d-wrvz5"]
Jul 24 23:32:33.158188 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.158159    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b6cccc4d-wrvz5"]
Jul 24 23:32:33.967786 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:32:33.967728    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0012a204-7a96-45a1-b91b-8b1baa931df5" path="/var/lib/kubelet/pods/0012a204-7a96-45a1-b91b-8b1baa931df5/volumes"
Jul 24 23:33:27.297760 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.297725    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z6pb7"]
Jul 24 23:33:27.298178 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.298009    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0012a204-7a96-45a1-b91b-8b1baa931df5" containerName="console"
Jul 24 23:33:27.298178 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.298030    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0012a204-7a96-45a1-b91b-8b1baa931df5" containerName="console"
Jul 24 23:33:27.298178 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.298083    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0012a204-7a96-45a1-b91b-8b1baa931df5" containerName="console"
Jul 24 23:33:27.301183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.301165    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.303421 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.303396    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 23:33:27.303546 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.303397    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jxfb8\""
Jul 24 23:33:27.303546 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.303396    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 23:33:27.309710 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.309683    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z6pb7"]
Jul 24 23:33:27.408947 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.408909    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95xt\" (UniqueName: \"kubernetes.io/projected/8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b-kube-api-access-n95xt\") pod \"cert-manager-55b4d954d5-z6pb7\" (UID: \"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b\") " pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.409118 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.408998    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b-bound-sa-token\") pod \"cert-manager-55b4d954d5-z6pb7\" (UID: \"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b\") " pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.509642 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.509600    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95xt\" (UniqueName: \"kubernetes.io/projected/8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b-kube-api-access-n95xt\") pod \"cert-manager-55b4d954d5-z6pb7\" (UID: \"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b\") " pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.509856 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.509694    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b-bound-sa-token\") pod \"cert-manager-55b4d954d5-z6pb7\" (UID: \"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b\") " pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.518472 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.518437    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b-bound-sa-token\") pod \"cert-manager-55b4d954d5-z6pb7\" (UID: \"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b\") " pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.518688 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.518665    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95xt\" (UniqueName: \"kubernetes.io/projected/8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b-kube-api-access-n95xt\") pod \"cert-manager-55b4d954d5-z6pb7\" (UID: \"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b\") " pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.623997 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.623904    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z6pb7"
Jul 24 23:33:27.752180 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:27.752087    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z6pb7"]
Jul 24 23:33:27.754813 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:33:27.754786    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8677a24f_b8e9_4dfb_ac4c_0fbb34bc151b.slice/crio-95d96767fafbd4ba55919128e64359bba68904960f2f170b083e3e2d0db5abbe WatchSource:0}: Error finding container 95d96767fafbd4ba55919128e64359bba68904960f2f170b083e3e2d0db5abbe: Status 404 returned error can't find the container with id 95d96767fafbd4ba55919128e64359bba68904960f2f170b083e3e2d0db5abbe
Jul 24 23:33:28.346373 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:28.346336    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z6pb7" event={"ID":"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b","Type":"ContainerStarted","Data":"95d96767fafbd4ba55919128e64359bba68904960f2f170b083e3e2d0db5abbe"}
Jul 24 23:33:47.429548 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:47.429442    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z6pb7" event={"ID":"8677a24f-b8e9-4dfb-ac4c-0fbb34bc151b","Type":"ContainerStarted","Data":"df753380ce86d4bcd95874253a983fb6cf7d6fa964d704b7e344114693b748d1"}
Jul 24 23:33:47.444682 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:47.444629    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-z6pb7" podStartSLOduration=1.819311141 podStartE2EDuration="20.444613346s" totalImagesPullingTime="18.625302205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:33:47.442891373 +0000 UTC m=+261.938603249" watchObservedRunningTime="2026-07-24 23:33:47.444613346 +0000 UTC m=+261.940325222"
Jul 24 23:33:52.740756 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.740722    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"]
Jul 24 23:33:52.744068 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.744050    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.746369 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.746340    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 23:33:52.746544 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.746447    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 23:33:52.746544 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.746475    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 23:33:52.746544 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.746526    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 23:33:52.746818 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.746801    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r9vz8\""
Jul 24 23:33:52.756321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.756292    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"]
Jul 24 23:33:52.833653 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.833615    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/823caa7f-30bd-478e-8590-92cb89fe2679-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.833653 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.833655    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhz9s\" (UniqueName: \"kubernetes.io/projected/823caa7f-30bd-478e-8590-92cb89fe2679-kube-api-access-qhz9s\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.833875 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.833683    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/823caa7f-30bd-478e-8590-92cb89fe2679-webhook-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.833875 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.833790    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/823caa7f-30bd-478e-8590-92cb89fe2679-tmp\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.833875 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.833848    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/823caa7f-30bd-478e-8590-92cb89fe2679-manifests\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.934935 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.934886    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/823caa7f-30bd-478e-8590-92cb89fe2679-tmp\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.934935 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.934946    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/823caa7f-30bd-478e-8590-92cb89fe2679-manifests\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.935170 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.934985    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/823caa7f-30bd-478e-8590-92cb89fe2679-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.935170 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.935007    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhz9s\" (UniqueName: \"kubernetes.io/projected/823caa7f-30bd-478e-8590-92cb89fe2679-kube-api-access-qhz9s\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.935170 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.935041    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/823caa7f-30bd-478e-8590-92cb89fe2679-webhook-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.935448 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.935423    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/823caa7f-30bd-478e-8590-92cb89fe2679-manifests\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.937364 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.937335    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/823caa7f-30bd-478e-8590-92cb89fe2679-tmp\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.937624 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.937604    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/823caa7f-30bd-478e-8590-92cb89fe2679-webhook-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.937697 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.937626    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/823caa7f-30bd-478e-8590-92cb89fe2679-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:52.946265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:52.946239    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhz9s\" (UniqueName: \"kubernetes.io/projected/823caa7f-30bd-478e-8590-92cb89fe2679-kube-api-access-qhz9s\") pod \"opendatahub-operator-controller-manager-69778f9898-tzlz8\" (UID: \"823caa7f-30bd-478e-8590-92cb89fe2679\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:53.054380 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:53.054341    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:53.225665 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:53.225635    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"]
Jul 24 23:33:53.226382 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:33:53.226343    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod823caa7f_30bd_478e_8590_92cb89fe2679.slice/crio-97ef0782be819cce3c6e8da98429dc5d218c8d6fda1e6449c523d6336eb118e3 WatchSource:0}: Error finding container 97ef0782be819cce3c6e8da98429dc5d218c8d6fda1e6449c523d6336eb118e3: Status 404 returned error can't find the container with id 97ef0782be819cce3c6e8da98429dc5d218c8d6fda1e6449c523d6336eb118e3
Jul 24 23:33:53.453802 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:53.453697    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8" event={"ID":"823caa7f-30bd-478e-8590-92cb89fe2679","Type":"ContainerStarted","Data":"97ef0782be819cce3c6e8da98429dc5d218c8d6fda1e6449c523d6336eb118e3"}
Jul 24 23:33:56.469353 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:56.469257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8" event={"ID":"823caa7f-30bd-478e-8590-92cb89fe2679","Type":"ContainerStarted","Data":"f316260b2da69c377c617e0ccc1456cc194fc2d6df1ee1f87cac24b603e7252b"}
Jul 24 23:33:57.474999 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:57.474966    2572 generic.go:356] "Generic (PLEG): container finished" podID="823caa7f-30bd-478e-8590-92cb89fe2679" containerID="f316260b2da69c377c617e0ccc1456cc194fc2d6df1ee1f87cac24b603e7252b" exitCode=0
Jul 24 23:33:57.475409 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:57.475009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8" event={"ID":"823caa7f-30bd-478e-8590-92cb89fe2679","Type":"ContainerDied","Data":"f316260b2da69c377c617e0ccc1456cc194fc2d6df1ee1f87cac24b603e7252b"}
Jul 24 23:33:58.481615 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:58.481582    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8" event={"ID":"823caa7f-30bd-478e-8590-92cb89fe2679","Type":"ContainerStarted","Data":"3de5158e6d1e68b5ded19866a04df74db68598f7021985c351d937ca974c4302"}
Jul 24 23:33:58.482058 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:58.481728    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:33:58.502641 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:33:58.502579    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8" podStartSLOduration=2.209183022 podStartE2EDuration="6.502562335s" totalImagesPullingTime="3.287911614s" totalInitContainerRuntime="1.005467699s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:33:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:33:58.500523869 +0000 UTC m=+272.996235744" watchObservedRunningTime="2026-07-24 23:33:58.502562335 +0000 UTC m=+272.998274210"
Jul 24 23:34:09.488275 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:09.488244    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-tzlz8"
Jul 24 23:34:12.056214 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.056182    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"]
Jul 24 23:34:12.067045 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.067018    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"]
Jul 24 23:34:12.067199 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.067113    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.070623 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.070592    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9b9ls\""
Jul 24 23:34:12.070779 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.070642    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 23:34:12.070779 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.070594    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 23:34:12.070779 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.070592    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 23:34:12.070779 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.070594    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 23:34:12.071018 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.070998    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 23:34:12.099381 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.099347    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9764f72c-6097-42a3-8e2b-cddd717eca45-manager-config\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.099381 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.099386    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4qfl\" (UniqueName: \"kubernetes.io/projected/9764f72c-6097-42a3-8e2b-cddd717eca45-kube-api-access-j4qfl\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.099619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.099414    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9764f72c-6097-42a3-8e2b-cddd717eca45-metrics-cert\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.099619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.099524    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9764f72c-6097-42a3-8e2b-cddd717eca45-cert\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.200723 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.200680    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9764f72c-6097-42a3-8e2b-cddd717eca45-manager-config\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.200918 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.200732    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4qfl\" (UniqueName: \"kubernetes.io/projected/9764f72c-6097-42a3-8e2b-cddd717eca45-kube-api-access-j4qfl\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.200918 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.200764    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9764f72c-6097-42a3-8e2b-cddd717eca45-metrics-cert\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.200918 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.200808    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9764f72c-6097-42a3-8e2b-cddd717eca45-cert\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.201361 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.201327    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9764f72c-6097-42a3-8e2b-cddd717eca45-manager-config\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.203271 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.203246    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9764f72c-6097-42a3-8e2b-cddd717eca45-metrics-cert\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.203348 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.203337    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9764f72c-6097-42a3-8e2b-cddd717eca45-cert\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.209677 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.209650    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4qfl\" (UniqueName: \"kubernetes.io/projected/9764f72c-6097-42a3-8e2b-cddd717eca45-kube-api-access-j4qfl\") pod \"lws-controller-manager-54fc5fd468-pkvjl\" (UID: \"9764f72c-6097-42a3-8e2b-cddd717eca45\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.378116 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.378028    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:12.510354 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.510317    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"]
Jul 24 23:34:12.513723 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:34:12.513689    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9764f72c_6097_42a3_8e2b_cddd717eca45.slice/crio-ceaa01d7b12bfc81402195173359f220ee0248691e0943232af62746df4dca9f WatchSource:0}: Error finding container ceaa01d7b12bfc81402195173359f220ee0248691e0943232af62746df4dca9f: Status 404 returned error can't find the container with id ceaa01d7b12bfc81402195173359f220ee0248691e0943232af62746df4dca9f
Jul 24 23:34:12.541328 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:12.541290    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl" event={"ID":"9764f72c-6097-42a3-8e2b-cddd717eca45","Type":"ContainerStarted","Data":"ceaa01d7b12bfc81402195173359f220ee0248691e0943232af62746df4dca9f"}
Jul 24 23:34:14.589111 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.589075    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"]
Jul 24 23:34:14.591642 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.591619    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.594333 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.594308    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 23:34:14.594485 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.594407    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 23:34:14.595251 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.595230    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-krzl2\""
Jul 24 23:34:14.595367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.595293    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 23:34:14.595628 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.595607    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 23:34:14.607335 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.607258    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"]
Jul 24 23:34:14.623732 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.623686    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tls-certs\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.623920 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.623761    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tmp\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.623920 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.623795    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp22v\" (UniqueName: \"kubernetes.io/projected/18e08dbc-3fb2-465d-9e6a-53f84b711c59-kube-api-access-jp22v\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.724604 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.724558    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tmp\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.724604 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.724610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp22v\" (UniqueName: \"kubernetes.io/projected/18e08dbc-3fb2-465d-9e6a-53f84b711c59-kube-api-access-jp22v\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.724859 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.724716    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tls-certs\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.724911 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:34:14.724879    2572 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 24 23:34:14.724991 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:34:14.724977    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tls-certs podName:18e08dbc-3fb2-465d-9e6a-53f84b711c59 nodeName:}" failed. No retries permitted until 2026-07-24 23:34:15.224954419 +0000 UTC m=+289.720666274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tls-certs") pod "kube-auth-proxy-6d7d7d465-8ctbx" (UID: "18e08dbc-3fb2-465d-9e6a-53f84b711c59") : secret "kube-auth-proxy-tls" not found
Jul 24 23:34:14.727348 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.727321    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tmp\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:14.735739 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:14.735672    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp22v\" (UniqueName: \"kubernetes.io/projected/18e08dbc-3fb2-465d-9e6a-53f84b711c59-kube-api-access-jp22v\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:15.230141 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:15.230046    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tls-certs\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:15.232516 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:15.232495    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e08dbc-3fb2-465d-9e6a-53f84b711c59-tls-certs\") pod \"kube-auth-proxy-6d7d7d465-8ctbx\" (UID: \"18e08dbc-3fb2-465d-9e6a-53f84b711c59\") " pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:15.505597 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:15.505563    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"
Jul 24 23:34:15.561636 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:15.561603    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl" event={"ID":"9764f72c-6097-42a3-8e2b-cddd717eca45","Type":"ContainerStarted","Data":"e65bf38e319c1d82448db3594a6b1b9693946f8931193f05f2637fc779b8f883"}
Jul 24 23:34:15.561800 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:15.561742    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:15.594642 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:15.594585    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl" podStartSLOduration=1.276058466 podStartE2EDuration="3.594565948s" totalImagesPullingTime="2.318507482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:34:15.593677846 +0000 UTC m=+290.089389735" watchObservedRunningTime="2026-07-24 23:34:15.594565948 +0000 UTC m=+290.090277823"
Jul 24 23:34:15.646399 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:15.646369    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx"]
Jul 24 23:34:15.648717 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:34:15.648686    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e08dbc_3fb2_465d_9e6a_53f84b711c59.slice/crio-f5631221dfba092893c0150f2fc362ad75c5e7a3397980a02eb4e12b8d058ee9 WatchSource:0}: Error finding container f5631221dfba092893c0150f2fc362ad75c5e7a3397980a02eb4e12b8d058ee9: Status 404 returned error can't find the container with id f5631221dfba092893c0150f2fc362ad75c5e7a3397980a02eb4e12b8d058ee9
Jul 24 23:34:16.567264 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:16.567220    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx" event={"ID":"18e08dbc-3fb2-465d-9e6a-53f84b711c59","Type":"ContainerStarted","Data":"f5631221dfba092893c0150f2fc362ad75c5e7a3397980a02eb4e12b8d058ee9"}
Jul 24 23:34:18.579990 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:18.579951    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx" event={"ID":"18e08dbc-3fb2-465d-9e6a-53f84b711c59","Type":"ContainerStarted","Data":"886f84d71850727abdae31f29204bd1ca017be1a75357c216ed91299afb2315f"}
Jul 24 23:34:18.599599 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:18.599539    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d7d7d465-8ctbx" podStartSLOduration=2.25602271 podStartE2EDuration="4.599520319s" totalImagesPullingTime="2.343497609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:34:18.598128114 +0000 UTC m=+293.093839991" watchObservedRunningTime="2026-07-24 23:34:18.599520319 +0000 UTC m=+293.095232194"
Jul 24 23:34:20.609778 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.609737    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"]
Jul 24 23:34:20.613219 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.613198    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.616084 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.616054    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6k79q\""
Jul 24 23:34:20.616351 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.616331    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 23:34:20.637648 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.637553    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"]
Jul 24 23:34:20.679252 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679206    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679446 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cplnj\" (UniqueName: \"kubernetes.io/projected/13347472-e20b-451d-9a70-2b2501dbe6d4-kube-api-access-cplnj\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679446 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679303    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679446 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679338    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13347472-e20b-451d-9a70-2b2501dbe6d4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679446 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679389    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679446 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679416    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679446 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679434    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679682 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679500    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.679682 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.679522    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780549 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780510    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780568    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780595    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cplnj\" (UniqueName: \"kubernetes.io/projected/13347472-e20b-451d-9a70-2b2501dbe6d4-kube-api-access-cplnj\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780618    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780651    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13347472-e20b-451d-9a70-2b2501dbe6d4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780710    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.780775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780734    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.781114 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.780787    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.781172 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.781154    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.781246 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.781213    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.781304 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.781280    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.781510 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.781486    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13347472-e20b-451d-9a70-2b2501dbe6d4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.781510 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.781495    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.783171 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.783136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.784005 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.783984    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.790104 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.790077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13347472-e20b-451d-9a70-2b2501dbe6d4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.790352 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.790327    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cplnj\" (UniqueName: \"kubernetes.io/projected/13347472-e20b-451d-9a70-2b2501dbe6d4-kube-api-access-cplnj\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cz94zk\" (UID: \"13347472-e20b-451d-9a70-2b2501dbe6d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:20.924537 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:20.924415    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:21.069105 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:21.069076    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"]
Jul 24 23:34:21.071523 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:34:21.071493    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13347472_e20b_451d_9a70_2b2501dbe6d4.slice/crio-31aeeabec610e70d60ee7bd15a3a9229f77a26bae205dce01faa5372b1135f43 WatchSource:0}: Error finding container 31aeeabec610e70d60ee7bd15a3a9229f77a26bae205dce01faa5372b1135f43: Status 404 returned error can't find the container with id 31aeeabec610e70d60ee7bd15a3a9229f77a26bae205dce01faa5372b1135f43
Jul 24 23:34:21.595480 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:21.595421    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk" event={"ID":"13347472-e20b-451d-9a70-2b2501dbe6d4","Type":"ContainerStarted","Data":"31aeeabec610e70d60ee7bd15a3a9229f77a26bae205dce01faa5372b1135f43"}
Jul 24 23:34:24.235868 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:24.235829    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:34:24.236181 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:24.235917    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:34:24.236181 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:24.235949    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:34:24.611277 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:24.611238    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk" event={"ID":"13347472-e20b-451d-9a70-2b2501dbe6d4","Type":"ContainerStarted","Data":"f41c1f04bdc0c0d9ee000ecae9217ea12a5115a3c4edb917a295f17713b27b99"}
Jul 24 23:34:24.638330 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:24.638268    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk" podStartSLOduration=1.476474632 podStartE2EDuration="4.638251019s" totalImagesPullingTime="3.161776387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:34:24.636201391 +0000 UTC m=+299.131913292" watchObservedRunningTime="2026-07-24 23:34:24.638251019 +0000 UTC m=+299.133962893"
Jul 24 23:34:24.925024 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:24.924933    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:25.864319 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:25.864291    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 23:34:25.929882 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:25.929852    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:26.570080 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:26.570045    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-pkvjl"
Jul 24 23:34:26.620594 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:26.620556    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:34:26.621654 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:34:26.621627    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cz94zk"
Jul 24 23:35:50.510621 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.510585    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b767d67fb-8zclz"]
Jul 24 23:35:50.513775 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.513752    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.522980 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.522953    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b767d67fb-8zclz"]
Jul 24 23:35:50.559654 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.559612    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-oauth-serving-cert\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.559857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.559701    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-console-config\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.559857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.559742    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-service-ca\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.559857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.559769    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cbsx\" (UniqueName: \"kubernetes.io/projected/5bd1eda3-6d18-4211-81da-ab5e71505234-kube-api-access-7cbsx\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.559857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.559817    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bd1eda3-6d18-4211-81da-ab5e71505234-console-oauth-config\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.559857 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.559856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bd1eda3-6d18-4211-81da-ab5e71505234-console-serving-cert\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.560067 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.559920    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-trusted-ca-bundle\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.661067 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661032    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cbsx\" (UniqueName: \"kubernetes.io/projected/5bd1eda3-6d18-4211-81da-ab5e71505234-kube-api-access-7cbsx\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.661265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661081    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bd1eda3-6d18-4211-81da-ab5e71505234-console-oauth-config\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.661265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661109    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bd1eda3-6d18-4211-81da-ab5e71505234-console-serving-cert\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.661265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661130    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-trusted-ca-bundle\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.661265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661146    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-oauth-serving-cert\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.661265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-console-config\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.661265 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661202    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-service-ca\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.662021 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.661993    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-console-config\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.662129 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.662087    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-trusted-ca-bundle\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.662129 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.662094    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-oauth-serving-cert\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.662207 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.662159    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd1eda3-6d18-4211-81da-ab5e71505234-service-ca\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.663735 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.663712    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bd1eda3-6d18-4211-81da-ab5e71505234-console-oauth-config\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.663869 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.663853    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bd1eda3-6d18-4211-81da-ab5e71505234-console-serving-cert\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.669846 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.669827    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cbsx\" (UniqueName: \"kubernetes.io/projected/5bd1eda3-6d18-4211-81da-ab5e71505234-kube-api-access-7cbsx\") pod \"console-b767d67fb-8zclz\" (UID: \"5bd1eda3-6d18-4211-81da-ab5e71505234\") " pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.823627 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.823532    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:35:50.961014 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.960988    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b767d67fb-8zclz"]
Jul 24 23:35:50.963815 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:35:50.963788    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd1eda3_6d18_4211_81da_ab5e71505234.slice/crio-e735f2b7e3f62e17b83d79f3f81d73a1c59d807517fc23f1b0f4849057cc6b2b WatchSource:0}: Error finding container e735f2b7e3f62e17b83d79f3f81d73a1c59d807517fc23f1b0f4849057cc6b2b: Status 404 returned error can't find the container with id e735f2b7e3f62e17b83d79f3f81d73a1c59d807517fc23f1b0f4849057cc6b2b
Jul 24 23:35:50.988514 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:50.988485    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b767d67fb-8zclz" event={"ID":"5bd1eda3-6d18-4211-81da-ab5e71505234","Type":"ContainerStarted","Data":"e735f2b7e3f62e17b83d79f3f81d73a1c59d807517fc23f1b0f4849057cc6b2b"}
Jul 24 23:35:51.999692 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:51.999660    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b767d67fb-8zclz" event={"ID":"5bd1eda3-6d18-4211-81da-ab5e71505234","Type":"ContainerStarted","Data":"143c1dd0a5ed90d46d3b70e24586b4d0d8ded2c5d9d363992c78051ed1a00222"}
Jul 24 23:35:52.018268 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:35:52.018208    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b767d67fb-8zclz" podStartSLOduration=2.018192515 podStartE2EDuration="2.018192515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:35:52.016327173 +0000 UTC m=+386.512039049" watchObservedRunningTime="2026-07-24 23:35:52.018192515 +0000 UTC m=+386.513904439"
Jul 24 23:36:00.554343 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.554297    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"]
Jul 24 23:36:00.558083 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.558059    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.560371 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.560350    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2bx2t\""
Jul 24 23:36:00.560513 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.560357    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 23:36:00.561247 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.561223    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 23:36:00.561346 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.561267    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 24 23:36:00.561346 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.561278    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 24 23:36:00.567720 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.567692    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"]
Jul 24 23:36:00.653569 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.653537    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b511136e-36dd-43d7-8588-baf1fa5bf10a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.653569 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.653573    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrmgn\" (UniqueName: \"kubernetes.io/projected/b511136e-36dd-43d7-8588-baf1fa5bf10a-kube-api-access-vrmgn\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.653794 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.653597    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b511136e-36dd-43d7-8588-baf1fa5bf10a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.754560 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.754520    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b511136e-36dd-43d7-8588-baf1fa5bf10a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.754560 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.754566    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrmgn\" (UniqueName: \"kubernetes.io/projected/b511136e-36dd-43d7-8588-baf1fa5bf10a-kube-api-access-vrmgn\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.754782 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.754593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b511136e-36dd-43d7-8588-baf1fa5bf10a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.755302 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.755282    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b511136e-36dd-43d7-8588-baf1fa5bf10a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.757073 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.757054    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b511136e-36dd-43d7-8588-baf1fa5bf10a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.765248 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.765223    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrmgn\" (UniqueName: \"kubernetes.io/projected/b511136e-36dd-43d7-8588-baf1fa5bf10a-kube-api-access-vrmgn\") pod \"kuadrant-console-plugin-66b5774985-rdt84\" (UID: \"b511136e-36dd-43d7-8588-baf1fa5bf10a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:00.824485 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.824362    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:36:00.824485 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.824429    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:36:00.829152 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.829127    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:36:00.868542 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:00.868497    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"
Jul 24 23:36:01.005467 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:01.005404    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84"]
Jul 24 23:36:01.008101 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:36:01.008073    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb511136e_36dd_43d7_8588_baf1fa5bf10a.slice/crio-718efa22c9b11c9c13ca0ddaa614b6709e826ec7ca82c5e230f8c6586821b693 WatchSource:0}: Error finding container 718efa22c9b11c9c13ca0ddaa614b6709e826ec7ca82c5e230f8c6586821b693: Status 404 returned error can't find the container with id 718efa22c9b11c9c13ca0ddaa614b6709e826ec7ca82c5e230f8c6586821b693
Jul 24 23:36:01.009419 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:01.009398    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:36:01.043823 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:01.043783    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84" event={"ID":"b511136e-36dd-43d7-8588-baf1fa5bf10a","Type":"ContainerStarted","Data":"718efa22c9b11c9c13ca0ddaa614b6709e826ec7ca82c5e230f8c6586821b693"}
Jul 24 23:36:01.047780 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:01.047757    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b767d67fb-8zclz"
Jul 24 23:36:01.089252 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:01.089177    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fb49894b-gtpb2"]
Jul 24 23:36:23.156614 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:23.156576    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84" event={"ID":"b511136e-36dd-43d7-8588-baf1fa5bf10a","Type":"ContainerStarted","Data":"e2b8ac98606cc952564f71786306b8bf42ec737070f004b506e3dd88f28e8d89"}
Jul 24 23:36:23.172214 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:23.172155    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rdt84" podStartSLOduration=1.658809317 podStartE2EDuration="23.172136663s" totalImagesPullingTime="21.513327346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:36:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:36:23.170406478 +0000 UTC m=+417.666118353" watchObservedRunningTime="2026-07-24 23:36:23.172136663 +0000 UTC m=+417.667848537"
Jul 24 23:36:26.108423 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.108377    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fb49894b-gtpb2" podUID="a5ae654c-dc12-40d3-bc44-04d1e55e95db" containerName="console" containerID="cri-o://9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725" gracePeriod=15
Jul 24 23:36:26.438796 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.438763    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:36:26.598637 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.598602    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-service-ca\") pod \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") "
Jul 24 23:36:26.598843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.598677    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-config\") pod \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") "
Jul 24 23:36:26.598843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.598730    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-trusted-ca-bundle\") pod \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") "
Jul 24 23:36:26.598843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.598754    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-oauth-config\") pod \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") "
Jul 24 23:36:26.598843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.598798    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl7w7\" (UniqueName: \"kubernetes.io/projected/a5ae654c-dc12-40d3-bc44-04d1e55e95db-kube-api-access-cl7w7\") pod \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") "
Jul 24 23:36:26.598843 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.598839    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-serving-cert\") pod \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") "
Jul 24 23:36:26.599101 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.598868    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-oauth-serving-cert\") pod \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\" (UID: \"a5ae654c-dc12-40d3-bc44-04d1e55e95db\") "
Jul 24 23:36:26.599180 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599145    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-service-ca" (OuterVolumeSpecName: "service-ca") pod "a5ae654c-dc12-40d3-bc44-04d1e55e95db" (UID: "a5ae654c-dc12-40d3-bc44-04d1e55e95db"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:36:26.599324 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599291    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a5ae654c-dc12-40d3-bc44-04d1e55e95db" (UID: "a5ae654c-dc12-40d3-bc44-04d1e55e95db"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:36:26.599475 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599299    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-config" (OuterVolumeSpecName: "console-config") pod "a5ae654c-dc12-40d3-bc44-04d1e55e95db" (UID: "a5ae654c-dc12-40d3-bc44-04d1e55e95db"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:36:26.599475 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599331    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a5ae654c-dc12-40d3-bc44-04d1e55e95db" (UID: "a5ae654c-dc12-40d3-bc44-04d1e55e95db"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:36:26.599619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599560    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-oauth-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:36:26.599619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599580    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:36:26.599619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599595    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:36:26.599619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.599607    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5ae654c-dc12-40d3-bc44-04d1e55e95db-trusted-ca-bundle\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:36:26.601060 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.601032    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a5ae654c-dc12-40d3-bc44-04d1e55e95db" (UID: "a5ae654c-dc12-40d3-bc44-04d1e55e95db"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:36:26.601060 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.601049    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a5ae654c-dc12-40d3-bc44-04d1e55e95db" (UID: "a5ae654c-dc12-40d3-bc44-04d1e55e95db"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:36:26.601227 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.601151    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ae654c-dc12-40d3-bc44-04d1e55e95db-kube-api-access-cl7w7" (OuterVolumeSpecName: "kube-api-access-cl7w7") pod "a5ae654c-dc12-40d3-bc44-04d1e55e95db" (UID: "a5ae654c-dc12-40d3-bc44-04d1e55e95db"). InnerVolumeSpecName "kube-api-access-cl7w7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:36:26.700166 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.700076    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl7w7\" (UniqueName: \"kubernetes.io/projected/a5ae654c-dc12-40d3-bc44-04d1e55e95db-kube-api-access-cl7w7\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:36:26.700166 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.700108    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-serving-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:36:26.700166 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:26.700118    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5ae654c-dc12-40d3-bc44-04d1e55e95db-console-oauth-config\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:36:27.182197 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.182157    2572 generic.go:356] "Generic (PLEG): container finished" podID="a5ae654c-dc12-40d3-bc44-04d1e55e95db" containerID="9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725" exitCode=0
Jul 24 23:36:27.182818 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.182206    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fb49894b-gtpb2"
Jul 24 23:36:27.182818 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.182245    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb49894b-gtpb2" event={"ID":"a5ae654c-dc12-40d3-bc44-04d1e55e95db","Type":"ContainerDied","Data":"9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725"}
Jul 24 23:36:27.182818 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.182300    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fb49894b-gtpb2" event={"ID":"a5ae654c-dc12-40d3-bc44-04d1e55e95db","Type":"ContainerDied","Data":"1a528355d98923bf49faca190e9a0d5b7b833f7c52b1d5894148f8092e7b99fe"}
Jul 24 23:36:27.182818 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.182321    2572 scope.go:117] "RemoveContainer" containerID="9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725"
Jul 24 23:36:27.191062 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.191043    2572 scope.go:117] "RemoveContainer" containerID="9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725"
Jul 24 23:36:27.191335 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:36:27.191317    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725\": container with ID starting with 9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725 not found: ID does not exist" containerID="9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725"
Jul 24 23:36:27.191385 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.191344    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725"} err="failed to get container status \"9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725\": rpc error: code = NotFound desc = could not find container \"9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725\": container with ID starting with 9651e55245cc09ba5b27706490dfdb704b1b5f78c79b2a978027961b738fe725 not found: ID does not exist"
Jul 24 23:36:27.203237 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.203179    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fb49894b-gtpb2"]
Jul 24 23:36:27.205168 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.205145    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fb49894b-gtpb2"]
Jul 24 23:36:27.965972 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:27.965932    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ae654c-dc12-40d3-bc44-04d1e55e95db" path="/var/lib/kubelet/pods/a5ae654c-dc12-40d3-bc44-04d1e55e95db/volumes"
Jul 24 23:36:40.474778 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.474741    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"]
Jul 24 23:36:40.475259 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.475059    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ae654c-dc12-40d3-bc44-04d1e55e95db" containerName="console"
Jul 24 23:36:40.475259 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.475070    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ae654c-dc12-40d3-bc44-04d1e55e95db" containerName="console"
Jul 24 23:36:40.475259 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.475151    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ae654c-dc12-40d3-bc44-04d1e55e95db" containerName="console"
Jul 24 23:36:40.492615 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.492584    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"]
Jul 24 23:36:40.492780 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.492707    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.495240 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.495213    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2rjzf\""
Jul 24 23:36:40.506733 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506699    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.506733 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.506944 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506755    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.506944 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506845    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.506944 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506883    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvd6m\" (UniqueName: \"kubernetes.io/projected/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-kube-api-access-vvd6m\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.506944 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506913    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.507075 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506951    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.507075 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.506998    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.507075 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.507023    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607644 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607607    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607854 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607657    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607854 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607854 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607854 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607746    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607854 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607770    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607854 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607816    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.607854 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607849    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvd6m\" (UniqueName: \"kubernetes.io/projected/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-kube-api-access-vvd6m\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.608282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.607886    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.608282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.608094    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.608282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.608179    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.608282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.608196    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.608547 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.608525    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.608711 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.608692    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.610601 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.610571    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.610734 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.610717    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.615967 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.615933    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.616566 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.616548    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvd6m\" (UniqueName: \"kubernetes.io/projected/0a038cf5-7cb5-47d1-88b6-c0ccef1051ec-kube-api-access-vvd6m\") pod \"maas-default-gateway-openshift-default-5ccb968c66-hm8ww\" (UID: \"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.802981 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.802944    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:40.945568 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.945535    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"]
Jul 24 23:36:40.948535 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:36:40.948508    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a038cf5_7cb5_47d1_88b6_c0ccef1051ec.slice/crio-3e91fd6081e6d4054b2c27ce8c2a42b2966cbedc9551af8d0e47ad32d9819c78 WatchSource:0}: Error finding container 3e91fd6081e6d4054b2c27ce8c2a42b2966cbedc9551af8d0e47ad32d9819c78: Status 404 returned error can't find the container with id 3e91fd6081e6d4054b2c27ce8c2a42b2966cbedc9551af8d0e47ad32d9819c78
Jul 24 23:36:40.950899 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.950860    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:36:40.950972 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.950953    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:36:40.951006 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:40.950992    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:36:41.246130 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:41.246089    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" event={"ID":"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec","Type":"ContainerStarted","Data":"3a9ff61cf5ca72c1089bbcadc2edc61c9e72d2181df758aae0659201692e1d76"}
Jul 24 23:36:41.246299 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:41.246156    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" event={"ID":"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec","Type":"ContainerStarted","Data":"3e91fd6081e6d4054b2c27ce8c2a42b2966cbedc9551af8d0e47ad32d9819c78"}
Jul 24 23:36:41.264541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:41.264478    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" podStartSLOduration=1.264434934 podStartE2EDuration="1.264434934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:36:41.26325123 +0000 UTC m=+435.758963116" watchObservedRunningTime="2026-07-24 23:36:41.264434934 +0000 UTC m=+435.760146813"
Jul 24 23:36:41.803818 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:41.803771    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:42.725522 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.725480    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:36:42.729017 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.728998    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:42.731141 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.731117    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 23:36:42.737929 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.737900    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:36:42.760416 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.760379    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:36:42.807783 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.807755    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:42.825515 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.825476    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0-config-file\") pod \"limitador-limitador-54d6d988d5-rmksn\" (UID: \"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:42.825710 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.825552    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5slq\" (UniqueName: \"kubernetes.io/projected/f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0-kube-api-access-n5slq\") pod \"limitador-limitador-54d6d988d5-rmksn\" (UID: \"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:42.926083 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.926042    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0-config-file\") pod \"limitador-limitador-54d6d988d5-rmksn\" (UID: \"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:42.926258 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.926101    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5slq\" (UniqueName: \"kubernetes.io/projected/f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0-kube-api-access-n5slq\") pod \"limitador-limitador-54d6d988d5-rmksn\" (UID: \"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:42.926747 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.926728    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0-config-file\") pod \"limitador-limitador-54d6d988d5-rmksn\" (UID: \"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:42.934093 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:42.934055    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5slq\" (UniqueName: \"kubernetes.io/projected/f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0-kube-api-access-n5slq\") pod \"limitador-limitador-54d6d988d5-rmksn\" (UID: \"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:43.040876 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.040834    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:43.174670 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.174647    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:36:43.177216 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:36:43.177188    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf419a8ba_1e71_42bd_a51a_e5ccf9e08cf0.slice/crio-1dc467c0d87bd8106535a681b64c930404dfedaa1d9d0bc32dde72262ded0702 WatchSource:0}: Error finding container 1dc467c0d87bd8106535a681b64c930404dfedaa1d9d0bc32dde72262ded0702: Status 404 returned error can't find the container with id 1dc467c0d87bd8106535a681b64c930404dfedaa1d9d0bc32dde72262ded0702
Jul 24 23:36:43.256360 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.256326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn" event={"ID":"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0","Type":"ContainerStarted","Data":"1dc467c0d87bd8106535a681b64c930404dfedaa1d9d0bc32dde72262ded0702"}
Jul 24 23:36:43.256584 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.256523    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:43.257586 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.257567    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:36:43.349865 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.349789    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-dxhct"]
Jul 24 23:36:43.354130 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.354113    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dxhct"
Jul 24 23:36:43.356335 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.356312    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d59d4\""
Jul 24 23:36:43.359081 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.359058    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dxhct"]
Jul 24 23:36:43.429689 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.429656    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtw8c\" (UniqueName: \"kubernetes.io/projected/b1265830-2f3c-4cef-9524-55ab2fde9127-kube-api-access-gtw8c\") pod \"authorino-75d556bd77-dxhct\" (UID: \"b1265830-2f3c-4cef-9524-55ab2fde9127\") " pod="kuadrant-system/authorino-75d556bd77-dxhct"
Jul 24 23:36:43.530294 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.530254    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtw8c\" (UniqueName: \"kubernetes.io/projected/b1265830-2f3c-4cef-9524-55ab2fde9127-kube-api-access-gtw8c\") pod \"authorino-75d556bd77-dxhct\" (UID: \"b1265830-2f3c-4cef-9524-55ab2fde9127\") " pod="kuadrant-system/authorino-75d556bd77-dxhct"
Jul 24 23:36:43.538465 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.538418    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtw8c\" (UniqueName: \"kubernetes.io/projected/b1265830-2f3c-4cef-9524-55ab2fde9127-kube-api-access-gtw8c\") pod \"authorino-75d556bd77-dxhct\" (UID: \"b1265830-2f3c-4cef-9524-55ab2fde9127\") " pod="kuadrant-system/authorino-75d556bd77-dxhct"
Jul 24 23:36:43.664826 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.664749    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dxhct"
Jul 24 23:36:43.847869 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:43.847607    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dxhct"]
Jul 24 23:36:43.850496 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:36:43.850446    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1265830_2f3c_4cef_9524_55ab2fde9127.slice/crio-bbb03f681a669fd4d5828ff0f1b1038edf95311c041d8b53cb7b38d4e5e01417 WatchSource:0}: Error finding container bbb03f681a669fd4d5828ff0f1b1038edf95311c041d8b53cb7b38d4e5e01417: Status 404 returned error can't find the container with id bbb03f681a669fd4d5828ff0f1b1038edf95311c041d8b53cb7b38d4e5e01417
Jul 24 23:36:44.262688 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:44.262649    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dxhct" event={"ID":"b1265830-2f3c-4cef-9524-55ab2fde9127","Type":"ContainerStarted","Data":"bbb03f681a669fd4d5828ff0f1b1038edf95311c041d8b53cb7b38d4e5e01417"}
Jul 24 23:36:48.285044 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:48.285002    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dxhct" event={"ID":"b1265830-2f3c-4cef-9524-55ab2fde9127","Type":"ContainerStarted","Data":"2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c"}
Jul 24 23:36:48.286672 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:48.286642    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn" event={"ID":"f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0","Type":"ContainerStarted","Data":"a2c2484b777d724cd65e6bf4bb5f03ac1358340ee89542cc12340b467909ee5e"}
Jul 24 23:36:48.286806 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:48.286786    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:36:48.299470 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:48.299408    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-dxhct" podStartSLOduration=1.547715525 podStartE2EDuration="5.299389534s" totalImagesPullingTime="3.751674009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:36:48.298444573 +0000 UTC m=+442.794156449" watchObservedRunningTime="2026-07-24 23:36:48.299389534 +0000 UTC m=+442.795101408"
Jul 24 23:36:48.313222 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:48.313161    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn" podStartSLOduration=1.8954802640000001 podStartE2EDuration="6.313142324s" totalImagesPullingTime="4.41766206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:36:48.312771985 +0000 UTC m=+442.808483871" watchObservedRunningTime="2026-07-24 23:36:48.313142324 +0000 UTC m=+442.808854199"
Jul 24 23:36:59.292645 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:36:59.292611    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rmksn"
Jul 24 23:37:18.449623 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:18.449587    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dxhct"]
Jul 24 23:37:18.450037 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:18.449807    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-dxhct" podUID="b1265830-2f3c-4cef-9524-55ab2fde9127" containerName="authorino" containerID="cri-o://2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c" gracePeriod=30
Jul 24 23:37:18.694336 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:18.694305    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dxhct"
Jul 24 23:37:18.856640 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:18.856596    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtw8c\" (UniqueName: \"kubernetes.io/projected/b1265830-2f3c-4cef-9524-55ab2fde9127-kube-api-access-gtw8c\") pod \"b1265830-2f3c-4cef-9524-55ab2fde9127\" (UID: \"b1265830-2f3c-4cef-9524-55ab2fde9127\") "
Jul 24 23:37:18.858903 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:18.858865    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1265830-2f3c-4cef-9524-55ab2fde9127-kube-api-access-gtw8c" (OuterVolumeSpecName: "kube-api-access-gtw8c") pod "b1265830-2f3c-4cef-9524-55ab2fde9127" (UID: "b1265830-2f3c-4cef-9524-55ab2fde9127"). InnerVolumeSpecName "kube-api-access-gtw8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:37:18.958321 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:18.958281    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtw8c\" (UniqueName: \"kubernetes.io/projected/b1265830-2f3c-4cef-9524-55ab2fde9127-kube-api-access-gtw8c\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:37:19.429320 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.429286    2572 generic.go:356] "Generic (PLEG): container finished" podID="b1265830-2f3c-4cef-9524-55ab2fde9127" containerID="2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c" exitCode=0
Jul 24 23:37:19.429535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.429332    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dxhct"
Jul 24 23:37:19.429535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.429354    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dxhct" event={"ID":"b1265830-2f3c-4cef-9524-55ab2fde9127","Type":"ContainerDied","Data":"2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c"}
Jul 24 23:37:19.429535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.429385    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dxhct" event={"ID":"b1265830-2f3c-4cef-9524-55ab2fde9127","Type":"ContainerDied","Data":"bbb03f681a669fd4d5828ff0f1b1038edf95311c041d8b53cb7b38d4e5e01417"}
Jul 24 23:37:19.429535 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.429401    2572 scope.go:117] "RemoveContainer" containerID="2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c"
Jul 24 23:37:19.438431 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.438410    2572 scope.go:117] "RemoveContainer" containerID="2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c"
Jul 24 23:37:19.438757 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:37:19.438738    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c\": container with ID starting with 2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c not found: ID does not exist" containerID="2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c"
Jul 24 23:37:19.438824 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.438766    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c"} err="failed to get container status \"2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c\": rpc error: code = NotFound desc = could not find container \"2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c\": container with ID starting with 2836c870533f6ca7f35de48e0764da8d02be9e05ffafecb685868e77e956517c not found: ID does not exist"
Jul 24 23:37:19.448906 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.448861    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dxhct"]
Jul 24 23:37:19.452441 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.452413    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dxhct"]
Jul 24 23:37:19.966309 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:19.966273    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1265830-2f3c-4cef-9524-55ab2fde9127" path="/var/lib/kubelet/pods/b1265830-2f3c-4cef-9524-55ab2fde9127/volumes"
Jul 24 23:37:25.887128 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:25.887088    2572 scope.go:117] "RemoveContainer" containerID="1e7ee756847517694ce9258d1bb05e267668bfd5efb34ad852d54ac82e0b3913"
Jul 24 23:37:59.091465 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:37:59.091414    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:38:06.645650 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:06.645554    2572 generic.go:356] "Generic (PLEG): container finished" podID="0a038cf5-7cb5-47d1-88b6-c0ccef1051ec" containerID="3a9ff61cf5ca72c1089bbcadc2edc61c9e72d2181df758aae0659201692e1d76" exitCode=0
Jul 24 23:38:06.645650 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:06.645628    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" event={"ID":"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec","Type":"ContainerDied","Data":"3a9ff61cf5ca72c1089bbcadc2edc61c9e72d2181df758aae0659201692e1d76"}
Jul 24 23:38:06.646205 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:06.646148    2572 scope.go:117] "RemoveContainer" containerID="3a9ff61cf5ca72c1089bbcadc2edc61c9e72d2181df758aae0659201692e1d76"
Jul 24 23:38:06.647128 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:06.647100    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:06.647230 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:06.647182    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:06.647290 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:06.647231    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 23:38:07.652037 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:07.652002    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" event={"ID":"0a038cf5-7cb5-47d1-88b6-c0ccef1051ec","Type":"ContainerStarted","Data":"5d82e0c49617c9e39193d583bb135426913873cc5651976e75305be1b801b23e"}
Jul 24 23:38:07.803949 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:07.803908    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:38:07.804188 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:07.804163    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-hm8ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 24 23:38:07.804271 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:07.804222    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" podUID="0a038cf5-7cb5-47d1-88b6-c0ccef1051ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 24 23:38:08.803576 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:08.803533    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-hm8ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 24 23:38:08.803977 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:08.803595    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" podUID="0a038cf5-7cb5-47d1-88b6-c0ccef1051ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 24 23:38:09.804030 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:09.803991    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-hm8ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 24 23:38:09.804438 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:09.804056    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" podUID="0a038cf5-7cb5-47d1-88b6-c0ccef1051ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 24 23:38:10.804024 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:10.803983    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-hm8ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 24 23:38:10.804201 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:10.804045    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww" podUID="0a038cf5-7cb5-47d1-88b6-c0ccef1051ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 24 23:38:11.807596 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:11.807559    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:38:11.808058 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:11.807837    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:38:11.808653 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:11.808635    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-hm8ww"
Jul 24 23:38:15.705738 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:15.705701    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:38:24.696398 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:24.696357    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:38:48.684920 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:38:48.684880    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:39:01.294603 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:01.294566    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:39:15.799796 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.799758    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"]
Jul 24 23:39:15.800193 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.800118    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1265830-2f3c-4cef-9524-55ab2fde9127" containerName="authorino"
Jul 24 23:39:15.800193 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.800131    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1265830-2f3c-4cef-9524-55ab2fde9127" containerName="authorino"
Jul 24 23:39:15.800193 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.800176    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1265830-2f3c-4cef-9524-55ab2fde9127" containerName="authorino"
Jul 24 23:39:15.803347 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.803328    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:15.805834 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.805802    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 23:39:15.806735 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.806717    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 24 23:39:15.806819 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.806721    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 23:39:15.806819 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.806769    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n27r8\""
Jul 24 23:39:15.812058 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.812030    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"]
Jul 24 23:39:15.916996 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.916959    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:15.916996 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.917015    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a986134b-78bf-4933-a201-1a25f130a257-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:15.917235 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.917081    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:15.917235 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.917144    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:15.917235 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.917167    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr5g2\" (UniqueName: \"kubernetes.io/projected/a986134b-78bf-4933-a201-1a25f130a257-kube-api-access-cr5g2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:15.917235 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:15.917193    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018128 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018090    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018128 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018134    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr5g2\" (UniqueName: \"kubernetes.io/projected/a986134b-78bf-4933-a201-1a25f130a257-kube-api-access-cr5g2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018198    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018237    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a986134b-78bf-4933-a201-1a25f130a257-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018410 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018272    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018654 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018602    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018654 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018637    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.018728 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.018681    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.020665 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.020640    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a986134b-78bf-4933-a201-1a25f130a257-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.020844 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.020829    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a986134b-78bf-4933-a201-1a25f130a257-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.026240 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.026204    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr5g2\" (UniqueName: \"kubernetes.io/projected/a986134b-78bf-4933-a201-1a25f130a257-kube-api-access-cr5g2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tvtwk\" (UID: \"a986134b-78bf-4933-a201-1a25f130a257\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.114975 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.114888    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:16.250086 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.250057    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"]
Jul 24 23:39:16.252130 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:39:16.252104    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda986134b_78bf_4933_a201_1a25f130a257.slice/crio-cd3c4e61006bf39855ae63b068b3873b908d6d76e9791a30c8b4f436e9ad12e8 WatchSource:0}: Error finding container cd3c4e61006bf39855ae63b068b3873b908d6d76e9791a30c8b4f436e9ad12e8: Status 404 returned error can't find the container with id cd3c4e61006bf39855ae63b068b3873b908d6d76e9791a30c8b4f436e9ad12e8
Jul 24 23:39:16.960073 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:16.960031    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk" event={"ID":"a986134b-78bf-4933-a201-1a25f130a257","Type":"ContainerStarted","Data":"cd3c4e61006bf39855ae63b068b3873b908d6d76e9791a30c8b4f436e9ad12e8"}
Jul 24 23:39:17.966926 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:17.966887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk" event={"ID":"a986134b-78bf-4933-a201-1a25f130a257","Type":"ContainerStarted","Data":"86a87ae02b20c9c37eeab29ffa4deb94bc5a507548c85484a3a73908052a66cc"}
Jul 24 23:39:17.983234 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:17.983180    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk" podStartSLOduration=2.013806169 podStartE2EDuration="2.983162976s" totalImagesPullingTime="969.356807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:39:17.982294193 +0000 UTC m=+592.478006068" watchObservedRunningTime="2026-07-24 23:39:17.983162976 +0000 UTC m=+592.478874869"
Jul 24 23:39:17.997738 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:17.997697    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:39:26.115642 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:26.115600    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:26.116120 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:26.115693    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:26.128212 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:26.128186    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:27.015822 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:27.015794    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tvtwk"
Jul 24 23:39:28.489706 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.489667    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"]
Jul 24 23:39:28.493188 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.493164    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.495447 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.495425    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 23:39:28.500844 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.500821    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"]
Jul 24 23:39:28.634322 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.634286    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.634519 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.634340    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9d10e94-717d-4b3d-840c-eda7fdeb717d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.634519 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.634425    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.634519 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.634479    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.634519 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.634501    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsxz6\" (UniqueName: \"kubernetes.io/projected/a9d10e94-717d-4b3d-840c-eda7fdeb717d-kube-api-access-dsxz6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.634689 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.634536    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.735578 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.735540    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.735782 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.735588    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9d10e94-717d-4b3d-840c-eda7fdeb717d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.735782 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.735623    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.735782 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.735643    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.735782 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.735662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsxz6\" (UniqueName: \"kubernetes.io/projected/a9d10e94-717d-4b3d-840c-eda7fdeb717d-kube-api-access-dsxz6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.735782 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.735696    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.736079 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.736055    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.736211 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.736095    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.736211 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.736120    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.738135 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.738114    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9d10e94-717d-4b3d-840c-eda7fdeb717d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.738219 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.738147    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9d10e94-717d-4b3d-840c-eda7fdeb717d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.743217 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.743161    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsxz6\" (UniqueName: \"kubernetes.io/projected/a9d10e94-717d-4b3d-840c-eda7fdeb717d-kube-api-access-dsxz6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-569mk\" (UID: \"a9d10e94-717d-4b3d-840c-eda7fdeb717d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.804057 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.804017    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:28.937971 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:28.937938    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"]
Jul 24 23:39:28.940915 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:39:28.940890    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9d10e94_717d_4b3d_840c_eda7fdeb717d.slice/crio-9ff5b534b431f9f60147b3dd1ed64e1a0515111a3924afe7775efd7b72260b0f WatchSource:0}: Error finding container 9ff5b534b431f9f60147b3dd1ed64e1a0515111a3924afe7775efd7b72260b0f: Status 404 returned error can't find the container with id 9ff5b534b431f9f60147b3dd1ed64e1a0515111a3924afe7775efd7b72260b0f
Jul 24 23:39:29.015122 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:29.015094    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk" event={"ID":"a9d10e94-717d-4b3d-840c-eda7fdeb717d","Type":"ContainerStarted","Data":"9ff5b534b431f9f60147b3dd1ed64e1a0515111a3924afe7775efd7b72260b0f"}
Jul 24 23:39:30.021252 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:30.021219    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk" event={"ID":"a9d10e94-717d-4b3d-840c-eda7fdeb717d","Type":"ContainerStarted","Data":"c5dfb857ebd32941aa1f18336e71f64394ecd01852613d17365000a3b2141f2e"}
Jul 24 23:39:30.037832 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:30.037776    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk" podStartSLOduration=1.847890197 podStartE2EDuration="2.03773477s" totalImagesPullingTime="189.844573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:39:30.036626869 +0000 UTC m=+604.532338757" watchObservedRunningTime="2026-07-24 23:39:30.03773477 +0000 UTC m=+604.533446645"
Jul 24 23:39:31.196570 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:31.196532    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:39:38.804140 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:38.804095    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:38.804613 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:38.804154    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:38.816497 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:38.816472    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:39:39.075825 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:39:39.075747    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-569mk"
Jul 24 23:41:01.371585 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.371545    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5688cf88c8-qn2hv"]
Jul 24 23:41:01.375023 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.375005    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.378090 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.378067    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d59d4\""
Jul 24 23:41:01.378203 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.378070    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 23:41:01.380859 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.380830    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5688cf88c8-qn2hv"]
Jul 24 23:41:01.503103 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.503065    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnd2f\" (UniqueName: \"kubernetes.io/projected/09eab24f-6c64-4613-b0e1-e13d290fd25b-kube-api-access-bnd2f\") pod \"authorino-5688cf88c8-qn2hv\" (UID: \"09eab24f-6c64-4613-b0e1-e13d290fd25b\") " pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.503275 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.503118    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09eab24f-6c64-4613-b0e1-e13d290fd25b-tls-cert\") pod \"authorino-5688cf88c8-qn2hv\" (UID: \"09eab24f-6c64-4613-b0e1-e13d290fd25b\") " pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.604411 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.604362    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnd2f\" (UniqueName: \"kubernetes.io/projected/09eab24f-6c64-4613-b0e1-e13d290fd25b-kube-api-access-bnd2f\") pod \"authorino-5688cf88c8-qn2hv\" (UID: \"09eab24f-6c64-4613-b0e1-e13d290fd25b\") " pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.604573 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.604432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09eab24f-6c64-4613-b0e1-e13d290fd25b-tls-cert\") pod \"authorino-5688cf88c8-qn2hv\" (UID: \"09eab24f-6c64-4613-b0e1-e13d290fd25b\") " pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.606836 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.606816    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09eab24f-6c64-4613-b0e1-e13d290fd25b-tls-cert\") pod \"authorino-5688cf88c8-qn2hv\" (UID: \"09eab24f-6c64-4613-b0e1-e13d290fd25b\") " pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.619387 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.619359    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnd2f\" (UniqueName: \"kubernetes.io/projected/09eab24f-6c64-4613-b0e1-e13d290fd25b-kube-api-access-bnd2f\") pod \"authorino-5688cf88c8-qn2hv\" (UID: \"09eab24f-6c64-4613-b0e1-e13d290fd25b\") " pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.685112 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.685026    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5688cf88c8-qn2hv"
Jul 24 23:41:01.816883 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.816846    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5688cf88c8-qn2hv"]
Jul 24 23:41:01.819953 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:41:01.819908    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09eab24f_6c64_4613_b0e1_e13d290fd25b.slice/crio-3f20bb665840a5041713bf10d155a25e40d53352a0a01dddfd66b11ff651696f WatchSource:0}: Error finding container 3f20bb665840a5041713bf10d155a25e40d53352a0a01dddfd66b11ff651696f: Status 404 returned error can't find the container with id 3f20bb665840a5041713bf10d155a25e40d53352a0a01dddfd66b11ff651696f
Jul 24 23:41:01.821642 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:01.821623    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:41:02.443989 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:02.443951    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5688cf88c8-qn2hv" event={"ID":"09eab24f-6c64-4613-b0e1-e13d290fd25b","Type":"ContainerStarted","Data":"6a5889b7471c8f6aa68bb14eeaee6b2efee60223d121f07cce4a3e4717507c06"}
Jul 24 23:41:02.444367 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:02.443996    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5688cf88c8-qn2hv" event={"ID":"09eab24f-6c64-4613-b0e1-e13d290fd25b","Type":"ContainerStarted","Data":"3f20bb665840a5041713bf10d155a25e40d53352a0a01dddfd66b11ff651696f"}
Jul 24 23:41:02.458092 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:02.458048    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5688cf88c8-qn2hv" podStartSLOduration=0.984841932 podStartE2EDuration="1.458031498s" totalImagesPullingTime="473.189566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:41:02.456935242 +0000 UTC m=+696.952647121" watchObservedRunningTime="2026-07-24 23:41:02.458031498 +0000 UTC m=+696.953743373"
Jul 24 23:41:30.589860 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:30.589818    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:41:41.186518 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:41.186474    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:41:52.188960 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:41:52.188913    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:43:30.086170 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:43:30.086129    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:43:41.087438 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:43:41.087394    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:44:17.985206 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:44:17.985123    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:44:27.192137 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:44:27.192101    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:44:37.492631 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:44:37.492593    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:44:45.987597 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:44:45.987557    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:44:55.089753 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:44:55.089711    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:45:06.488566 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:45:06.488525    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:45:16.489832 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:45:16.489795    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:45:26.187271 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:45:26.187236    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:45:51.681937 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:45:51.681855    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:46:00.681083 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:46:00.681042    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:46:22.890765 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:46:22.890728    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:46:34.188966 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:46:34.188929    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:46:51.299793 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:46:51.299751    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:47:00.193470 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:47:00.193413    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:47:09.791001 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:47:09.790964    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:47:19.081552 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:47:19.081467    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:47:28.287399 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:47:28.287360    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:49:16.990821 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:49:16.990781    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:49:28.290644 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:49:28.290602    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:49:36.988861 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:49:36.988820    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:49:49.196582 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:49:49.196542    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:49:59.990337 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:49:59.990299    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:50:12.091824 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:50:12.091787    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:50:20.005721 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:50:20.005617    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:50:30.983031 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:50:30.982992    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:50:41.284157 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:50:41.284117    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:50:51.390689 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:50:51.390650    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:51:03.992174 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:03.992136    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:51:28.395191 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:28.395150    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:51:36.393823 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:36.393785    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:51:39.394671 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.394635    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"]
Jul 24 23:51:39.397541 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.397524    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.399703 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.399682    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-bb5774f2-kserve-self-signed-certs\""
Jul 24 23:51:39.412554 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.412524    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"]
Jul 24 23:51:39.514663 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.514632    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tmp-dir\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.514663 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.514666    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7k4z\" (UniqueName: \"kubernetes.io/projected/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-kube-api-access-n7k4z\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.514895 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.514725    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-model-cache\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.514895 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.514772    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tls-certs\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.514895 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.514789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-home\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.514895 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.514810    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-dshm\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.615982 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.615939    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-model-cache\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616191 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616001    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tls-certs\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616191 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616040    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-home\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616191 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616072    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-dshm\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616191 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616132    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tmp-dir\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616191 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7k4z\" (UniqueName: \"kubernetes.io/projected/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-kube-api-access-n7k4z\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616443 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616412    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-home\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616443 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616424    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-model-cache\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.616585 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.616524    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tmp-dir\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.618372 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.618351    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-dshm\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.618549 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.618529    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tls-certs\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.623864 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.623844    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7k4z\" (UniqueName: \"kubernetes.io/projected/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-kube-api-access-n7k4z\") pod \"e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") " pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.706650 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.706563    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:39.836033 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:39.835998    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"]
Jul 24 23:51:39.838010 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:51:39.837986    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f9b72ff_dca0_4f3a_bc69_b4d363c1123c.slice/crio-843ce336f58bba829de6fc763d24327f4ee1998bd1e1762b735f62b2d71b5f95 WatchSource:0}: Error finding container 843ce336f58bba829de6fc763d24327f4ee1998bd1e1762b735f62b2d71b5f95: Status 404 returned error can't find the container with id 843ce336f58bba829de6fc763d24327f4ee1998bd1e1762b735f62b2d71b5f95
Jul 24 23:51:40.473246 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:40.473206    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266" event={"ID":"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c","Type":"ContainerStarted","Data":"08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f"}
Jul 24 23:51:40.473651 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:40.473255    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266" event={"ID":"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c","Type":"ContainerStarted","Data":"843ce336f58bba829de6fc763d24327f4ee1998bd1e1762b735f62b2d71b5f95"}
Jul 24 23:51:40.489016 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:40.488957    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266" podStartSLOduration=1.488939538 podStartE2EDuration="1.488939538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:51:40.487346077 +0000 UTC m=+1334.983057952" watchObservedRunningTime="2026-07-24 23:51:40.488939538 +0000 UTC m=+1334.984651416"
Jul 24 23:51:48.290128 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.290036    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"]
Jul 24 23:51:48.298548 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.298519    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.301106 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.301066    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-bb5774f2-kserve-self-signed-certs\""
Jul 24 23:51:48.303886 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.303861    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"]
Jul 24 23:51:48.399915 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.399878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-home\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.399915 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.399918    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-dshm\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.400148 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.399992    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-tmp-dir\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.400148 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.400078    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-694ft\" (UniqueName: \"kubernetes.io/projected/43f35a37-00c4-488a-bd40-41966483df12-kube-api-access-694ft\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.400148 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.400132    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-model-cache\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.400274 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.400151    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43f35a37-00c4-488a-bd40-41966483df12-tls-certs\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501271 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501234    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-model-cache\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501271 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501276    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43f35a37-00c4-488a-bd40-41966483df12-tls-certs\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501534 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501409    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-home\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501534 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501444    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-dshm\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501624 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501534    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-tmp-dir\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501676 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501657    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-694ft\" (UniqueName: \"kubernetes.io/projected/43f35a37-00c4-488a-bd40-41966483df12-kube-api-access-694ft\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501729 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501682    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-model-cache\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501817 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501793    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-home\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.501955 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.501924    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-tmp-dir\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.503722 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.503703    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-dshm\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.503890 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.503875    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43f35a37-00c4-488a-bd40-41966483df12-tls-certs\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.509183 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.509158    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-694ft\" (UniqueName: \"kubernetes.io/projected/43f35a37-00c4-488a-bd40-41966483df12-kube-api-access-694ft\") pod \"e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") " pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.610132 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.610036    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:48.746101 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:48.745965    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"]
Jul 24 23:51:48.748316 ip-10-0-141-134 kubenswrapper[2572]: W0724 23:51:48.748286    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43f35a37_00c4_488a_bd40_41966483df12.slice/crio-2c85c858349f5383959ea32da1311d961beabe7cfdc336719ba323f5def76912 WatchSource:0}: Error finding container 2c85c858349f5383959ea32da1311d961beabe7cfdc336719ba323f5def76912: Status 404 returned error can't find the container with id 2c85c858349f5383959ea32da1311d961beabe7cfdc336719ba323f5def76912
Jul 24 23:51:49.518999 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:49.518964    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl" event={"ID":"43f35a37-00c4-488a-bd40-41966483df12","Type":"ContainerStarted","Data":"5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736"}
Jul 24 23:51:49.519372 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:49.519007    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl" event={"ID":"43f35a37-00c4-488a-bd40-41966483df12","Type":"ContainerStarted","Data":"2c85c858349f5383959ea32da1311d961beabe7cfdc336719ba323f5def76912"}
Jul 24 23:51:49.536619 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:49.536562    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl" podStartSLOduration=1.5365455799999999 podStartE2EDuration="1.53654558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:51:49.534995862 +0000 UTC m=+1344.030707738" watchObservedRunningTime="2026-07-24 23:51:49.53654558 +0000 UTC m=+1344.032257504"
Jul 24 23:51:49.707361 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:49.707259    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:49.707591 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:49.707542    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:49.720556 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:49.720525    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:50.535355 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:50.535325    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:51:51.890266 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:51.890233    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:51:58.611190 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:58.611146    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:58.611190 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:58.611205    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:58.623749 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:58.623724    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:51:59.582141 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:51:59.582104    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:52:25.157705 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:25.157670    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"]
Jul 24 23:52:26.465558 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:26.465518    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"]
Jul 24 23:52:29.489393 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:29.489355    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:52:40.176690 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.176646    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266" podUID="5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" containerName="main" containerID="cri-o://08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f" gracePeriod=45
Jul 24 23:52:40.436536 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.436442    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:52:40.495168 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495126    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-dshm\") pod \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") "
Jul 24 23:52:40.495168 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495177    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7k4z\" (UniqueName: \"kubernetes.io/projected/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-kube-api-access-n7k4z\") pod \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") "
Jul 24 23:52:40.495428 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495210    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-home\") pod \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") "
Jul 24 23:52:40.495428 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495232    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tls-certs\") pod \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") "
Jul 24 23:52:40.495428 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495279    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tmp-dir\") pod \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") "
Jul 24 23:52:40.495428 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495295    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-model-cache\") pod \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\" (UID: \"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c\") "
Jul 24 23:52:40.495648 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495570    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-home" (OuterVolumeSpecName: "home") pod "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" (UID: "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:40.495648 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495598    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" (UID: "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:40.495770 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495648    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-model-cache" (OuterVolumeSpecName: "model-cache") pod "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" (UID: "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:40.495770 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495730    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:40.495770 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.495749    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:40.497556 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.497532    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-kube-api-access-n7k4z" (OuterVolumeSpecName: "kube-api-access-n7k4z") pod "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" (UID: "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c"). InnerVolumeSpecName "kube-api-access-n7k4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:52:40.497556 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.497547    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" (UID: "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:52:40.497661 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.497595    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-dshm" (OuterVolumeSpecName: "dshm") pod "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" (UID: "5f9b72ff-dca0-4f3a-bc69-b4d363c1123c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:40.596808 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.596771    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:40.596808 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.596799    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:40.596808 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.596808    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7k4z\" (UniqueName: \"kubernetes.io/projected/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-kube-api-access-n7k4z\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:40.596808 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.596818    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:40.763643 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.763602    2572 generic.go:356] "Generic (PLEG): container finished" podID="5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" containerID="08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f" exitCode=0
Jul 24 23:52:40.763844 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.763655    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"
Jul 24 23:52:40.763844 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.763654    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266" event={"ID":"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c","Type":"ContainerDied","Data":"08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f"}
Jul 24 23:52:40.763844 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.763771    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266" event={"ID":"5f9b72ff-dca0-4f3a-bc69-b4d363c1123c","Type":"ContainerDied","Data":"843ce336f58bba829de6fc763d24327f4ee1998bd1e1762b735f62b2d71b5f95"}
Jul 24 23:52:40.763844 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.763788    2572 scope.go:117] "RemoveContainer" containerID="08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f"
Jul 24 23:52:40.772533 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.772486    2572 scope.go:117] "RemoveContainer" containerID="08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f"
Jul 24 23:52:40.772757 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:52:40.772739    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f\": container with ID starting with 08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f not found: ID does not exist" containerID="08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f"
Jul 24 23:52:40.772821 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.772765    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f"} err="failed to get container status \"08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f\": rpc error: code = NotFound desc = could not find container \"08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f\": container with ID starting with 08e4e1c32be5f1aa40f22b8304bae496fe10d712d8a802a91921034103e3e60f not found: ID does not exist"
Jul 24 23:52:40.783598 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.783572    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"]
Jul 24 23:52:40.789704 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:40.789683    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-bb5774f2-kserve-6dbdb85f59-pc266"]
Jul 24 23:52:41.484978 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.484935    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl" podUID="43f35a37-00c4-488a-bd40-41966483df12" containerName="main" containerID="cri-o://5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736" gracePeriod=45
Jul 24 23:52:41.730788 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.730760    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:52:41.772172 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.772133    2572 generic.go:356] "Generic (PLEG): container finished" podID="43f35a37-00c4-488a-bd40-41966483df12" containerID="5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736" exitCode=0
Jul 24 23:52:41.772336 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.772179    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"
Jul 24 23:52:41.772336 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.772213    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl" event={"ID":"43f35a37-00c4-488a-bd40-41966483df12","Type":"ContainerDied","Data":"5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736"}
Jul 24 23:52:41.772336 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.772256    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl" event={"ID":"43f35a37-00c4-488a-bd40-41966483df12","Type":"ContainerDied","Data":"2c85c858349f5383959ea32da1311d961beabe7cfdc336719ba323f5def76912"}
Jul 24 23:52:41.772336 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.772281    2572 scope.go:117] "RemoveContainer" containerID="5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736"
Jul 24 23:52:41.780499 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.780479    2572 scope.go:117] "RemoveContainer" containerID="5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736"
Jul 24 23:52:41.780765 ip-10-0-141-134 kubenswrapper[2572]: E0724 23:52:41.780739    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736\": container with ID starting with 5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736 not found: ID does not exist" containerID="5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736"
Jul 24 23:52:41.780873 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.780767    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736"} err="failed to get container status \"5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736\": rpc error: code = NotFound desc = could not find container \"5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736\": container with ID starting with 5342481deb3b4413635ec6505d59f9eec16b8315e3296f1b39e485df64b38736 not found: ID does not exist"
Jul 24 23:52:41.807393 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807363    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-model-cache\") pod \"43f35a37-00c4-488a-bd40-41966483df12\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") "
Jul 24 23:52:41.807393 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807402    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-tmp-dir\") pod \"43f35a37-00c4-488a-bd40-41966483df12\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") "
Jul 24 23:52:41.807661 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807472    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-home\") pod \"43f35a37-00c4-488a-bd40-41966483df12\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") "
Jul 24 23:52:41.807661 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807497    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-694ft\" (UniqueName: \"kubernetes.io/projected/43f35a37-00c4-488a-bd40-41966483df12-kube-api-access-694ft\") pod \"43f35a37-00c4-488a-bd40-41966483df12\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") "
Jul 24 23:52:41.807661 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807534    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43f35a37-00c4-488a-bd40-41966483df12-tls-certs\") pod \"43f35a37-00c4-488a-bd40-41966483df12\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") "
Jul 24 23:52:41.807661 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807553    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-dshm\") pod \"43f35a37-00c4-488a-bd40-41966483df12\" (UID: \"43f35a37-00c4-488a-bd40-41966483df12\") "
Jul 24 23:52:41.807860 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807717    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-model-cache" (OuterVolumeSpecName: "model-cache") pod "43f35a37-00c4-488a-bd40-41966483df12" (UID: "43f35a37-00c4-488a-bd40-41966483df12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:41.807860 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807725    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43f35a37-00c4-488a-bd40-41966483df12" (UID: "43f35a37-00c4-488a-bd40-41966483df12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:41.807860 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807807    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-home" (OuterVolumeSpecName: "home") pod "43f35a37-00c4-488a-bd40-41966483df12" (UID: "43f35a37-00c4-488a-bd40-41966483df12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:41.808027 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807892    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:41.808027 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.807912    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:41.809785 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.809758    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-dshm" (OuterVolumeSpecName: "dshm") pod "43f35a37-00c4-488a-bd40-41966483df12" (UID: "43f35a37-00c4-488a-bd40-41966483df12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:52:41.809895 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.809807    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43f35a37-00c4-488a-bd40-41966483df12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43f35a37-00c4-488a-bd40-41966483df12" (UID: "43f35a37-00c4-488a-bd40-41966483df12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:52:41.809997 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.809980    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43f35a37-00c4-488a-bd40-41966483df12-kube-api-access-694ft" (OuterVolumeSpecName: "kube-api-access-694ft") pod "43f35a37-00c4-488a-bd40-41966483df12" (UID: "43f35a37-00c4-488a-bd40-41966483df12"). InnerVolumeSpecName "kube-api-access-694ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:52:41.908917 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.908880    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:41.908917 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.908914    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-694ft\" (UniqueName: \"kubernetes.io/projected/43f35a37-00c4-488a-bd40-41966483df12-kube-api-access-694ft\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:41.908917 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.908925    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43f35a37-00c4-488a-bd40-41966483df12-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:41.909144 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.908934    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43f35a37-00c4-488a-bd40-41966483df12-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 24 23:52:41.966780 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:41.966749    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" path="/var/lib/kubelet/pods/5f9b72ff-dca0-4f3a-bc69-b4d363c1123c/volumes"
Jul 24 23:52:42.086844 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:42.086743    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"]
Jul 24 23:52:42.088977 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:42.088952    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-bb5774f2-kserve-6fff5fd979-5s8gl"]
Jul 24 23:52:42.691434 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:42.691400    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:52:43.966098 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:43.966062    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43f35a37-00c4-488a-bd40-41966483df12" path="/var/lib/kubelet/pods/43f35a37-00c4-488a-bd40-41966483df12/volumes"
Jul 24 23:52:52.498797 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:52:52.498734    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:53:01.483010 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:53:01.482973    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:53:13.889973 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:53:13.889917    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:53:22.690702 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:53:22.690611    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:53:33.503150 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:53:33.503112    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:53:41.691780 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:53:41.691745    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:53:51.082667 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:53:51.082630    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:54:03.394171 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:54:03.394133    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:54:13.594641 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:54:13.594600    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:54:22.491420 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:54:22.491383    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:54:33.592050 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:54:33.592012    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:54:46.084241 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:54:46.084201    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:54:54.989266 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:54:54.989179    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:55:05.891324 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:55:05.891276    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:55:16.587229 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:55:16.587192    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:55:26.287282 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:55:26.287246    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:55:39.490960 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:55:39.490920    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:55:49.284023 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:55:49.283975    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:55:58.587681 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:55:58.587639    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:56:10.587629 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:56:10.587588    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:56:43.888651 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:56:43.888565    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:56:52.692013 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:56:52.691966    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:57:28.286216 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:57:28.286177    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:57:37.990795 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:57:37.990754    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:58:12.284336 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:58:12.284293    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:58:25.081543 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:58:25.081496    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 24 23:58:56.381266 ip-10-0-141-134 kubenswrapper[2572]: I0724 23:58:56.381232    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:00:22.795699 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:00:22.795659    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:00:33.589720 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:00:33.589684    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:00:44.689833 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:00:44.689793    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:01:03.485517 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:01:03.485477    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:01:38.588772 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:01:38.588734    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:01:59.890946 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:01:59.890910    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:02:15.589581 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:02:15.589541    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:02:37.289368 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:02:37.289275    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:02:45.797710 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:02:45.797669    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:03:45.086678 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:03:45.086626    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:03:53.186125 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:03:53.186017    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:04:38.391628 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:38.391584    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:04:49.816951 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.816906    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"]
Jul 25 00:04:49.817382 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.817365    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" containerName="main"
Jul 25 00:04:49.817426 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.817384    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" containerName="main"
Jul 25 00:04:49.817426 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.817400    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43f35a37-00c4-488a-bd40-41966483df12" containerName="main"
Jul 25 00:04:49.817426 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.817405    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="43f35a37-00c4-488a-bd40-41966483df12" containerName="main"
Jul 25 00:04:49.817541 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.817486    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f9b72ff-dca0-4f3a-bc69-b4d363c1123c" containerName="main"
Jul 25 00:04:49.817541 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.817495    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="43f35a37-00c4-488a-bd40-41966483df12" containerName="main"
Jul 25 00:04:49.820128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.820111    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:49.823655 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.823629    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-820f12\"/\"default-dockercfg-27gjl\""
Jul 25 00:04:49.823846 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.823629    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-820f12\"/\"kube-root-ca.crt\""
Jul 25 00:04:49.823947 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.823679    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-820f12\"/\"test-model-820f12-kserve-self-signed-certs\""
Jul 25 00:04:49.824027 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.823724    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-820f12\"/\"openshift-service-ca.crt\""
Jul 25 00:04:49.829303 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.829277    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"]
Jul 25 00:04:49.935481 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.935430    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09efbaaf-901a-4ced-857b-2be0769df03b-tls-certs\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:49.935669 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.935495    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-model-cache\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:49.935669 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.935513    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-home\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:49.935669 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.935598    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-dshm\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:49.935669 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.935632    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrd7m\" (UniqueName: \"kubernetes.io/projected/09efbaaf-901a-4ced-857b-2be0769df03b-kube-api-access-qrd7m\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:49.935869 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:49.935765    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-tmp-dir\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.036747 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.036700    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-tmp-dir\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.036936 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.036766    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09efbaaf-901a-4ced-857b-2be0769df03b-tls-certs\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.036936 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.036790    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-model-cache\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.036936 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.036809    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-home\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.036936 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.036845    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-dshm\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.037139 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.036937    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrd7m\" (UniqueName: \"kubernetes.io/projected/09efbaaf-901a-4ced-857b-2be0769df03b-kube-api-access-qrd7m\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.037206 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.037181    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-tmp-dir\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.037291 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.037266    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-model-cache\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.037347 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.037272    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-home\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.039200 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.039176    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-dshm\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.039468 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.039438    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09efbaaf-901a-4ced-857b-2be0769df03b-tls-certs\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.044426 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.044398    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrd7m\" (UniqueName: \"kubernetes.io/projected/09efbaaf-901a-4ced-857b-2be0769df03b-kube-api-access-qrd7m\") pod \"test-model-820f12-kserve-56676bdc6c-zjrdk\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") " pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.131526 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.131404    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:04:50.277041 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:50.276994    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"]
Jul 25 00:04:50.278568 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:04:50.278530    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09efbaaf_901a_4ced_857b_2be0769df03b.slice/crio-c3654f070c244d321a5faa7cfd34de8ab763508eea39b1595d187e84a9bbca00 WatchSource:0}: Error finding container c3654f070c244d321a5faa7cfd34de8ab763508eea39b1595d187e84a9bbca00: Status 404 returned error can't find the container with id c3654f070c244d321a5faa7cfd34de8ab763508eea39b1595d187e84a9bbca00
Jul 25 00:04:51.184192 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:51.184156    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk" event={"ID":"09efbaaf-901a-4ced-857b-2be0769df03b","Type":"ContainerStarted","Data":"70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113"}
Jul 25 00:04:51.184683 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:51.184199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk" event={"ID":"09efbaaf-901a-4ced-857b-2be0769df03b","Type":"ContainerStarted","Data":"c3654f070c244d321a5faa7cfd34de8ab763508eea39b1595d187e84a9bbca00"}
Jul 25 00:04:51.199846 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:51.199785    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk" podStartSLOduration=2.199767723 podStartE2EDuration="2.199767723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:04:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:04:51.198417295 +0000 UTC m=+2125.694129174" watchObservedRunningTime="2026-07-25 00:04:51.199767723 +0000 UTC m=+2125.695479598"
Jul 25 00:04:55.286835 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:04:55.286791    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:05:00.132324 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:00.132275    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:05:00.132821 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:00.132345    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:05:00.145670 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:00.145635    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:05:00.238354 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:00.238321    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:05:12.889958 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.889918    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"]
Jul 25 00:05:12.893286 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.893256    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:12.896752 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.896723    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-107131\"/\"default-dockercfg-sc45m\""
Jul 25 00:05:12.896926 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.896719    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-107131\"/\"openshift-service-ca.crt\""
Jul 25 00:05:12.896926 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.896730    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-107131\"/\"kube-root-ca.crt\""
Jul 25 00:05:12.896926 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.896731    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-107131\"/\"test-model-107131-kserve-self-signed-certs\""
Jul 25 00:05:12.903179 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.903147    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"]
Jul 25 00:05:12.964752 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.964714    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-home\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:12.964752 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.964756    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-dshm\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:12.965007 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.964788    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrkls\" (UniqueName: \"kubernetes.io/projected/f64d125b-229c-4a94-ada1-18cc07102fb1-kube-api-access-jrkls\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:12.965007 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.964875    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-tmp-dir\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:12.965007 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.964954    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-model-cache\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:12.965007 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:12.964993    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d125b-229c-4a94-ada1-18cc07102fb1-tls-certs\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066118 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066072    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-home\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066118 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066116    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-dshm\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066360 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066149    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrkls\" (UniqueName: \"kubernetes.io/projected/f64d125b-229c-4a94-ada1-18cc07102fb1-kube-api-access-jrkls\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066360 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066220    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-tmp-dir\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066360 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066326    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-model-cache\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066527 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066384    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d125b-229c-4a94-ada1-18cc07102fb1-tls-certs\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066664 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066627    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-home\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066781 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066714    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-tmp-dir\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.066843 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.066824    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-model-cache\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.068538 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.068518    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-dshm\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.068814 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.068797    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d125b-229c-4a94-ada1-18cc07102fb1-tls-certs\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.074272 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.074244    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrkls\" (UniqueName: \"kubernetes.io/projected/f64d125b-229c-4a94-ada1-18cc07102fb1-kube-api-access-jrkls\") pod \"test-model-107131-kserve-5945bb5fbc-pnk97\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") " pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.205563 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.205443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:13.352124 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:13.352055    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"]
Jul 25 00:05:13.356179 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:05:13.356144    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf64d125b_229c_4a94_ada1_18cc07102fb1.slice/crio-caa73e48425ad55e50d64fa327408f89c1605368c6aa2515ddaa03ab1be1dbd0 WatchSource:0}: Error finding container caa73e48425ad55e50d64fa327408f89c1605368c6aa2515ddaa03ab1be1dbd0: Status 404 returned error can't find the container with id caa73e48425ad55e50d64fa327408f89c1605368c6aa2515ddaa03ab1be1dbd0
Jul 25 00:05:14.295263 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:14.295224    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97" event={"ID":"f64d125b-229c-4a94-ada1-18cc07102fb1","Type":"ContainerStarted","Data":"49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83"}
Jul 25 00:05:14.295263 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:14.295271    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97" event={"ID":"f64d125b-229c-4a94-ada1-18cc07102fb1","Type":"ContainerStarted","Data":"caa73e48425ad55e50d64fa327408f89c1605368c6aa2515ddaa03ab1be1dbd0"}
Jul 25 00:05:14.311756 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:14.311695    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97" podStartSLOduration=2.311676586 podStartE2EDuration="2.311676586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:05:14.309751908 +0000 UTC m=+2148.805463782" watchObservedRunningTime="2026-07-25 00:05:14.311676586 +0000 UTC m=+2148.807388462"
Jul 25 00:05:17.194196 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:17.194157    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:05:23.206484 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:23.206415    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:23.206920 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:23.206744    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:23.219325 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:23.219296    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:23.351860 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:23.351831    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:05:31.091836 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:31.091795    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:05:54.780187 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:54.780150    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"]
Jul 25 00:05:58.692291 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:05:58.692246    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:06:05.880361 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:05.880320    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"]
Jul 25 00:06:09.800676 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:09.800601    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk" podUID="09efbaaf-901a-4ced-857b-2be0769df03b" containerName="main" containerID="cri-o://70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113" gracePeriod=45
Jul 25 00:06:10.057437 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.057350    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:06:10.227792 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.227732    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-tmp-dir\") pod \"09efbaaf-901a-4ced-857b-2be0769df03b\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") "
Jul 25 00:06:10.227792 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.227801    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-dshm\") pod \"09efbaaf-901a-4ced-857b-2be0769df03b\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") "
Jul 25 00:06:10.228031 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.227827    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-model-cache\") pod \"09efbaaf-901a-4ced-857b-2be0769df03b\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") "
Jul 25 00:06:10.228031 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.227888    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrd7m\" (UniqueName: \"kubernetes.io/projected/09efbaaf-901a-4ced-857b-2be0769df03b-kube-api-access-qrd7m\") pod \"09efbaaf-901a-4ced-857b-2be0769df03b\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") "
Jul 25 00:06:10.228031 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.227945    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09efbaaf-901a-4ced-857b-2be0769df03b-tls-certs\") pod \"09efbaaf-901a-4ced-857b-2be0769df03b\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") "
Jul 25 00:06:10.228031 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.228014    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-home\") pod \"09efbaaf-901a-4ced-857b-2be0769df03b\" (UID: \"09efbaaf-901a-4ced-857b-2be0769df03b\") "
Jul 25 00:06:10.228224 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.228100    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09efbaaf-901a-4ced-857b-2be0769df03b" (UID: "09efbaaf-901a-4ced-857b-2be0769df03b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:10.228224 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.228203    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-model-cache" (OuterVolumeSpecName: "model-cache") pod "09efbaaf-901a-4ced-857b-2be0769df03b" (UID: "09efbaaf-901a-4ced-857b-2be0769df03b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:10.228390 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.228369    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-home" (OuterVolumeSpecName: "home") pod "09efbaaf-901a-4ced-857b-2be0769df03b" (UID: "09efbaaf-901a-4ced-857b-2be0769df03b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:10.228477 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.228417    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:10.228477 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.228440    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:10.230377 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.230342    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-dshm" (OuterVolumeSpecName: "dshm") pod "09efbaaf-901a-4ced-857b-2be0769df03b" (UID: "09efbaaf-901a-4ced-857b-2be0769df03b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:10.230377 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.230357    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09efbaaf-901a-4ced-857b-2be0769df03b-kube-api-access-qrd7m" (OuterVolumeSpecName: "kube-api-access-qrd7m") pod "09efbaaf-901a-4ced-857b-2be0769df03b" (UID: "09efbaaf-901a-4ced-857b-2be0769df03b"). InnerVolumeSpecName "kube-api-access-qrd7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:06:10.230562 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.230406    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09efbaaf-901a-4ced-857b-2be0769df03b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09efbaaf-901a-4ced-857b-2be0769df03b" (UID: "09efbaaf-901a-4ced-857b-2be0769df03b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:06:10.329338 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.329228    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:10.329338 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.329280    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09efbaaf-901a-4ced-857b-2be0769df03b-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:10.329338 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.329291    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrd7m\" (UniqueName: \"kubernetes.io/projected/09efbaaf-901a-4ced-857b-2be0769df03b-kube-api-access-qrd7m\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:10.329338 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.329301    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09efbaaf-901a-4ced-857b-2be0769df03b-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:10.384699 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.384650    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:06:10.562533 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.562492    2572 generic.go:356] "Generic (PLEG): container finished" podID="09efbaaf-901a-4ced-857b-2be0769df03b" containerID="70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113" exitCode=0
Jul 25 00:06:10.562705 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.562540    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk" event={"ID":"09efbaaf-901a-4ced-857b-2be0769df03b","Type":"ContainerDied","Data":"70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113"}
Jul 25 00:06:10.562705 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.562542    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"
Jul 25 00:06:10.562705 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.562570    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk" event={"ID":"09efbaaf-901a-4ced-857b-2be0769df03b","Type":"ContainerDied","Data":"c3654f070c244d321a5faa7cfd34de8ab763508eea39b1595d187e84a9bbca00"}
Jul 25 00:06:10.562705 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.562587    2572 scope.go:117] "RemoveContainer" containerID="70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113"
Jul 25 00:06:10.571794 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.571774    2572 scope.go:117] "RemoveContainer" containerID="70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113"
Jul 25 00:06:10.572092 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:06:10.572070    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113\": container with ID starting with 70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113 not found: ID does not exist" containerID="70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113"
Jul 25 00:06:10.572138 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.572099    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113"} err="failed to get container status \"70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113\": rpc error: code = NotFound desc = could not find container \"70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113\": container with ID starting with 70ce3ec41b9879c568b53bb6263b11fba1fe46e46229b4984724530aefa94113 not found: ID does not exist"
Jul 25 00:06:10.583074 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.582996    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"]
Jul 25 00:06:10.584775 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:10.584751    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-820f12/test-model-820f12-kserve-56676bdc6c-zjrdk"]
Jul 25 00:06:11.966645 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:11.966607    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09efbaaf-901a-4ced-857b-2be0769df03b" path="/var/lib/kubelet/pods/09efbaaf-901a-4ced-857b-2be0769df03b/volumes"
Jul 25 00:06:19.490975 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:19.490935    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:06:20.900418 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:20.900349    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97" podUID="f64d125b-229c-4a94-ada1-18cc07102fb1" containerName="main" containerID="cri-o://49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83" gracePeriod=45
Jul 25 00:06:21.160421 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.160344    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:06:21.240163 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240119    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrkls\" (UniqueName: \"kubernetes.io/projected/f64d125b-229c-4a94-ada1-18cc07102fb1-kube-api-access-jrkls\") pod \"f64d125b-229c-4a94-ada1-18cc07102fb1\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") "
Jul 25 00:06:21.240336 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240209    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-model-cache\") pod \"f64d125b-229c-4a94-ada1-18cc07102fb1\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") "
Jul 25 00:06:21.240379 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240350    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-dshm\") pod \"f64d125b-229c-4a94-ada1-18cc07102fb1\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") "
Jul 25 00:06:21.240495 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240479    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-tmp-dir\") pod \"f64d125b-229c-4a94-ada1-18cc07102fb1\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") "
Jul 25 00:06:21.240537 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240490    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-model-cache" (OuterVolumeSpecName: "model-cache") pod "f64d125b-229c-4a94-ada1-18cc07102fb1" (UID: "f64d125b-229c-4a94-ada1-18cc07102fb1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:21.240576 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240539    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-home\") pod \"f64d125b-229c-4a94-ada1-18cc07102fb1\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") "
Jul 25 00:06:21.240620 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240571    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d125b-229c-4a94-ada1-18cc07102fb1-tls-certs\") pod \"f64d125b-229c-4a94-ada1-18cc07102fb1\" (UID: \"f64d125b-229c-4a94-ada1-18cc07102fb1\") "
Jul 25 00:06:21.240732 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240705    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f64d125b-229c-4a94-ada1-18cc07102fb1" (UID: "f64d125b-229c-4a94-ada1-18cc07102fb1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:21.240818 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240796    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-home" (OuterVolumeSpecName: "home") pod "f64d125b-229c-4a94-ada1-18cc07102fb1" (UID: "f64d125b-229c-4a94-ada1-18cc07102fb1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:21.240977 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240959    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:21.241118 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240982    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:21.241118 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.240996    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:21.242724 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.242690    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f64d125b-229c-4a94-ada1-18cc07102fb1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f64d125b-229c-4a94-ada1-18cc07102fb1" (UID: "f64d125b-229c-4a94-ada1-18cc07102fb1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:06:21.242724 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.242714    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-dshm" (OuterVolumeSpecName: "dshm") pod "f64d125b-229c-4a94-ada1-18cc07102fb1" (UID: "f64d125b-229c-4a94-ada1-18cc07102fb1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:06:21.242724 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.242693    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64d125b-229c-4a94-ada1-18cc07102fb1-kube-api-access-jrkls" (OuterVolumeSpecName: "kube-api-access-jrkls") pod "f64d125b-229c-4a94-ada1-18cc07102fb1" (UID: "f64d125b-229c-4a94-ada1-18cc07102fb1"). InnerVolumeSpecName "kube-api-access-jrkls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:06:21.342142 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.342103    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d125b-229c-4a94-ada1-18cc07102fb1-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:21.342142 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.342135    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d125b-229c-4a94-ada1-18cc07102fb1-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:21.342142 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.342147    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrkls\" (UniqueName: \"kubernetes.io/projected/f64d125b-229c-4a94-ada1-18cc07102fb1-kube-api-access-jrkls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:06:21.617924 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.617883    2572 generic.go:356] "Generic (PLEG): container finished" podID="f64d125b-229c-4a94-ada1-18cc07102fb1" containerID="49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83" exitCode=0
Jul 25 00:06:21.618128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.617935    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"
Jul 25 00:06:21.618128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.617973    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97" event={"ID":"f64d125b-229c-4a94-ada1-18cc07102fb1","Type":"ContainerDied","Data":"49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83"}
Jul 25 00:06:21.618128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.618015    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97" event={"ID":"f64d125b-229c-4a94-ada1-18cc07102fb1","Type":"ContainerDied","Data":"caa73e48425ad55e50d64fa327408f89c1605368c6aa2515ddaa03ab1be1dbd0"}
Jul 25 00:06:21.618128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.618033    2572 scope.go:117] "RemoveContainer" containerID="49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83"
Jul 25 00:06:21.631064 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.631038    2572 scope.go:117] "RemoveContainer" containerID="49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83"
Jul 25 00:06:21.631393 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:06:21.631368    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83\": container with ID starting with 49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83 not found: ID does not exist" containerID="49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83"
Jul 25 00:06:21.631441 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.631405    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83"} err="failed to get container status \"49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83\": rpc error: code = NotFound desc = could not find container \"49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83\": container with ID starting with 49f7649bb13c6a763d2b3a486e3e8136cf82feda78f703dd58e32ebbbd6e5e83 not found: ID does not exist"
Jul 25 00:06:21.639881 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.639809    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"]
Jul 25 00:06:21.641740 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.641707    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-107131/test-model-107131-kserve-5945bb5fbc-pnk97"]
Jul 25 00:06:21.967123 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:21.967034    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64d125b-229c-4a94-ada1-18cc07102fb1" path="/var/lib/kubelet/pods/f64d125b-229c-4a94-ada1-18cc07102fb1/volumes"
Jul 25 00:06:41.289558 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:41.289515    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:06:58.189217 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:06:58.189125    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:07:14.153061 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.153022    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"]
Jul 25 00:07:14.153430 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.153402    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09efbaaf-901a-4ced-857b-2be0769df03b" containerName="main"
Jul 25 00:07:14.153430 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.153414    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="09efbaaf-901a-4ced-857b-2be0769df03b" containerName="main"
Jul 25 00:07:14.153430 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.153425    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64d125b-229c-4a94-ada1-18cc07102fb1" containerName="main"
Jul 25 00:07:14.153430 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.153431    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64d125b-229c-4a94-ada1-18cc07102fb1" containerName="main"
Jul 25 00:07:14.153647 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.153506    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="09efbaaf-901a-4ced-857b-2be0769df03b" containerName="main"
Jul 25 00:07:14.153647 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.153514    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64d125b-229c-4a94-ada1-18cc07102fb1" containerName="main"
Jul 25 00:07:14.156639 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.156614    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"
Jul 25 00:07:14.159086 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.159060    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:07:14.159254 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.159124    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 00:07:14.162743 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.162710    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"]
Jul 25 00:07:14.230837 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.230793    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9\" (UID: \"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"
Jul 25 00:07:14.332283 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.332241    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9\" (UID: \"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"
Jul 25 00:07:14.332965 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.332943    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9\" (UID: \"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"
Jul 25 00:07:14.469420 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.469316    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"
Jul 25 00:07:14.629128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.629004    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"]
Jul 25 00:07:14.632011 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:07:14.631968    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ab7ab6_e73f_4cbb_905b_7e81d6ce4ede.slice/crio-cee85277ea95f721c7c9f7be5577114f32bd1b130d89afbde7c2ae3bdebd1648 WatchSource:0}: Error finding container cee85277ea95f721c7c9f7be5577114f32bd1b130d89afbde7c2ae3bdebd1648: Status 404 returned error can't find the container with id cee85277ea95f721c7c9f7be5577114f32bd1b130d89afbde7c2ae3bdebd1648
Jul 25 00:07:14.633893 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.633876    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:07:14.874202 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:14.874166    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9" event={"ID":"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede","Type":"ContainerStarted","Data":"cee85277ea95f721c7c9f7be5577114f32bd1b130d89afbde7c2ae3bdebd1648"}
Jul 25 00:07:15.696677 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:15.696634    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:07:23.919073 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:23.919037    2572 generic.go:356] "Generic (PLEG): container finished" podID="94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede" containerID="5e873ab6c00ef18d360ed51e7ecf10557d524bb971d67395007936c0b545765e" exitCode=0
Jul 25 00:07:23.919499 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:23.919130    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9" event={"ID":"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede","Type":"ContainerDied","Data":"5e873ab6c00ef18d360ed51e7ecf10557d524bb971d67395007936c0b545765e"}
Jul 25 00:07:25.059617 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:25.059590    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"
Jul 25 00:07:25.144966 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:25.144922    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede-maas-api-service-ca\") pod \"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede\" (UID: \"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede\") "
Jul 25 00:07:25.145302 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:25.145279    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede" (UID: "94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:07:25.245665 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:25.245633    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:07:25.929783 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:25.929751    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"
Jul 25 00:07:25.929783 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:25.929759    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9" event={"ID":"94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede","Type":"ContainerDied","Data":"cee85277ea95f721c7c9f7be5577114f32bd1b130d89afbde7c2ae3bdebd1648"}
Jul 25 00:07:25.930223 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:25.929805    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cee85277ea95f721c7c9f7be5577114f32bd1b130d89afbde7c2ae3bdebd1648"
Jul 25 00:07:44.235675 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:44.235630    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"]
Jul 25 00:07:44.237424 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:44.237398    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-97aaee-l9zd9"]
Jul 25 00:07:45.966798 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:45.966764    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede" path="/var/lib/kubelet/pods/94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede/volumes"
Jul 25 00:07:46.086644 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:07:46.086606    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:08:11.086885 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:11.086844    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:08:26.391259 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:26.391163    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:08:42.589748 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:42.589709    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:08:44.608187 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.608149    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"]
Jul 25 00:08:44.608703 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.608552    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede" containerName="revoke-keys"
Jul 25 00:08:44.608703 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.608565    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede" containerName="revoke-keys"
Jul 25 00:08:44.608703 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.608644    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="94ab7ab6-e73f-4cbb-905b-7e81d6ce4ede" containerName="revoke-keys"
Jul 25 00:08:44.611727 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.611700    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.614164 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.614137    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"openshift-service-ca.crt\""
Jul 25 00:08:44.615082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.615058    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"default-dockercfg-mqhrp\""
Jul 25 00:08:44.615082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.615073    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"auth-test-model-92df89-kserve-self-signed-certs\""
Jul 25 00:08:44.615258 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.615118    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"kube-root-ca.crt\""
Jul 25 00:08:44.622022 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.621991    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"]
Jul 25 00:08:44.719777 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.719735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-home\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.719997 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.719800    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-model-cache\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.719997 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.719843    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-dshm\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.719997 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.719921    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c5adcda-4852-4b4b-bcac-591ca4125c49-tls-certs\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.720122 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.719991    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjc8g\" (UniqueName: \"kubernetes.io/projected/9c5adcda-4852-4b4b-bcac-591ca4125c49-kube-api-access-tjc8g\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.720122 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.720052    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-tmp-dir\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.821594 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.821552    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c5adcda-4852-4b4b-bcac-591ca4125c49-tls-certs\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.821764 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.821612    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjc8g\" (UniqueName: \"kubernetes.io/projected/9c5adcda-4852-4b4b-bcac-591ca4125c49-kube-api-access-tjc8g\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.821764 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.821649    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-tmp-dir\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.821764 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.821680    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-home\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.821764 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.821748    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-model-cache\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.821961 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.821930    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-dshm\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.822157 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.822135    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-tmp-dir\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.822236 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.822155    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-home\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.822236 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.822185    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-model-cache\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.824132 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.824100    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-dshm\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.824314 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.824299    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c5adcda-4852-4b4b-bcac-591ca4125c49-tls-certs\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.829098 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.829066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjc8g\" (UniqueName: \"kubernetes.io/projected/9c5adcda-4852-4b4b-bcac-591ca4125c49-kube-api-access-tjc8g\") pod \"auth-test-model-92df89-kserve-5f6d7694f9-plm9b\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") " pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:44.923105 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:44.922999    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:45.069710 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:45.069678    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"]
Jul 25 00:08:45.072020 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:08:45.071982    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c5adcda_4852_4b4b_bcac_591ca4125c49.slice/crio-2a9ba7328a8fed9c4508aa89fb947d9df1046f3d8e23b57c16f0145bdf2c7c73 WatchSource:0}: Error finding container 2a9ba7328a8fed9c4508aa89fb947d9df1046f3d8e23b57c16f0145bdf2c7c73: Status 404 returned error can't find the container with id 2a9ba7328a8fed9c4508aa89fb947d9df1046f3d8e23b57c16f0145bdf2c7c73
Jul 25 00:08:45.321575 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:45.321528    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b" event={"ID":"9c5adcda-4852-4b4b-bcac-591ca4125c49","Type":"ContainerStarted","Data":"45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527"}
Jul 25 00:08:45.321764 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:45.321586    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b" event={"ID":"9c5adcda-4852-4b4b-bcac-591ca4125c49","Type":"ContainerStarted","Data":"2a9ba7328a8fed9c4508aa89fb947d9df1046f3d8e23b57c16f0145bdf2c7c73"}
Jul 25 00:08:45.337200 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:45.337136    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b" podStartSLOduration=1.3371174780000001 podStartE2EDuration="1.337117478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:08:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:08:45.335381291 +0000 UTC m=+2359.831093176" watchObservedRunningTime="2026-07-25 00:08:45.337117478 +0000 UTC m=+2359.832829403"
Jul 25 00:08:52.791813 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:52.791771    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:08:54.923412 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:54.923375    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:54.923939 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:54.923440    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:54.935946 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:54.935913    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:08:55.380255 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:08:55.380221    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:09:02.085809 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.085773    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"]
Jul 25 00:09:02.089083 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.089055    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.092591 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.092555    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"default-dockercfg-gzckw\""
Jul 25 00:09:02.092755 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.092599    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"auth-test-model-413844-kserve-self-signed-certs\""
Jul 25 00:09:02.092755 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.092618    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"kube-root-ca.crt\""
Jul 25 00:09:02.092755 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.092618    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"openshift-service-ca.crt\""
Jul 25 00:09:02.098533 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.098505    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"]
Jul 25 00:09:02.194957 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.194914    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-home\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.195139 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.194972    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-dshm\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.195139 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.195061    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7aae092-85e5-476f-8831-3ad835aafa26-tls-certs\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.195139 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.195105    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-tmp-dir\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.195139 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.195123    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrvth\" (UniqueName: \"kubernetes.io/projected/a7aae092-85e5-476f-8831-3ad835aafa26-kube-api-access-rrvth\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.195281 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.195164    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-model-cache\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.296668 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.296621    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-tmp-dir\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.296668 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.296664    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrvth\" (UniqueName: \"kubernetes.io/projected/a7aae092-85e5-476f-8831-3ad835aafa26-kube-api-access-rrvth\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.296937 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.296688    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-model-cache\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.296937 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.296752    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-home\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.296937 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.296791    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-dshm\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.296937 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.296836    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7aae092-85e5-476f-8831-3ad835aafa26-tls-certs\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.297156 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.297100    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-home\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.297156 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.297117    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-tmp-dir\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.297253 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.297177    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-model-cache\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.299155 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.299130    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-dshm\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.299295 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.299278    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7aae092-85e5-476f-8831-3ad835aafa26-tls-certs\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.304845 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.304813    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrvth\" (UniqueName: \"kubernetes.io/projected/a7aae092-85e5-476f-8831-3ad835aafa26-kube-api-access-rrvth\") pod \"auth-test-model-413844-kserve-54fc5cc495-7df25\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") " pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.401370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.401279    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:02.547500 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:02.547443    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"]
Jul 25 00:09:02.548996 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:09:02.548966    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7aae092_85e5_476f_8831_3ad835aafa26.slice/crio-ee719eb69b69c6053fd996c4eca25f7e4103198989c6bf19121b953129b7811c WatchSource:0}: Error finding container ee719eb69b69c6053fd996c4eca25f7e4103198989c6bf19121b953129b7811c: Status 404 returned error can't find the container with id ee719eb69b69c6053fd996c4eca25f7e4103198989c6bf19121b953129b7811c
Jul 25 00:09:03.407527 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:03.407492    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25" event={"ID":"a7aae092-85e5-476f-8831-3ad835aafa26","Type":"ContainerStarted","Data":"ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99"}
Jul 25 00:09:03.407926 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:03.407537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25" event={"ID":"a7aae092-85e5-476f-8831-3ad835aafa26","Type":"ContainerStarted","Data":"ee719eb69b69c6053fd996c4eca25f7e4103198989c6bf19121b953129b7811c"}
Jul 25 00:09:03.424756 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:03.424702    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25" podStartSLOduration=1.424686146 podStartE2EDuration="1.424686146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:09:03.42329878 +0000 UTC m=+2377.919010652" watchObservedRunningTime="2026-07-25 00:09:03.424686146 +0000 UTC m=+2377.920398021"
Jul 25 00:09:05.893561 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:05.893522    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:09:12.402006 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:12.401966    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:12.402435 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:12.402228    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:12.414751 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:12.414723    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:12.462689 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:12.462658    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:09:29.884823 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:29.884786    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:09:50.682858 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:09:50.682765    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:10:02.704651 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:10:02.704607    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:10:14.886444 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:10:14.886400    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:10:35.289736 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:10:35.289690    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:10:53.693194 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:10:53.693147    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:11:15.895728 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:11:15.895690    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:11:32.385095 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:11:32.385014    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:11:59.803161 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:11:59.803117    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:12:16.791608 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:12:16.791570    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:12:42.083036 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:12:42.082997    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:12:59.089405 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:12:59.089316    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:13:21.208284 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:21.208239    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:13:24.370598 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:24.370560    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"]
Jul 25 00:13:25.186861 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:25.186821    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"]
Jul 25 00:13:26.191392 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:26.191354    2572 scope.go:117] "RemoveContainer" containerID="5e873ab6c00ef18d360ed51e7ecf10557d524bb971d67395007936c0b545765e"
Jul 25 00:13:36.794695 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:36.794659    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:13:39.390029 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.389987    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b" podUID="9c5adcda-4852-4b4b-bcac-591ca4125c49" containerName="main" containerID="cri-o://45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527" gracePeriod=45
Jul 25 00:13:39.649841 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.649757    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:13:39.734772 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.734734    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjc8g\" (UniqueName: \"kubernetes.io/projected/9c5adcda-4852-4b4b-bcac-591ca4125c49-kube-api-access-tjc8g\") pod \"9c5adcda-4852-4b4b-bcac-591ca4125c49\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") "
Jul 25 00:13:39.734952 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.734808    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-tmp-dir\") pod \"9c5adcda-4852-4b4b-bcac-591ca4125c49\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") "
Jul 25 00:13:39.734952 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.734866    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c5adcda-4852-4b4b-bcac-591ca4125c49-tls-certs\") pod \"9c5adcda-4852-4b4b-bcac-591ca4125c49\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") "
Jul 25 00:13:39.734952 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.734913    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-home\") pod \"9c5adcda-4852-4b4b-bcac-591ca4125c49\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") "
Jul 25 00:13:39.735130 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.734951    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-dshm\") pod \"9c5adcda-4852-4b4b-bcac-591ca4125c49\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") "
Jul 25 00:13:39.735130 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.734983    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-model-cache\") pod \"9c5adcda-4852-4b4b-bcac-591ca4125c49\" (UID: \"9c5adcda-4852-4b4b-bcac-591ca4125c49\") "
Jul 25 00:13:39.735130 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.735099    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9c5adcda-4852-4b4b-bcac-591ca4125c49" (UID: "9c5adcda-4852-4b4b-bcac-591ca4125c49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:39.735298 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.735228    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-home" (OuterVolumeSpecName: "home") pod "9c5adcda-4852-4b4b-bcac-591ca4125c49" (UID: "9c5adcda-4852-4b4b-bcac-591ca4125c49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:39.735352 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.735302    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-model-cache" (OuterVolumeSpecName: "model-cache") pod "9c5adcda-4852-4b4b-bcac-591ca4125c49" (UID: "9c5adcda-4852-4b4b-bcac-591ca4125c49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:39.735432 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.735413    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:39.735533 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.735437    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:39.735533 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.735483    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:39.737344 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.737307    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-dshm" (OuterVolumeSpecName: "dshm") pod "9c5adcda-4852-4b4b-bcac-591ca4125c49" (UID: "9c5adcda-4852-4b4b-bcac-591ca4125c49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:39.737501 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.737403    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c5adcda-4852-4b4b-bcac-591ca4125c49-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9c5adcda-4852-4b4b-bcac-591ca4125c49" (UID: "9c5adcda-4852-4b4b-bcac-591ca4125c49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:13:39.737501 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.737414    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5adcda-4852-4b4b-bcac-591ca4125c49-kube-api-access-tjc8g" (OuterVolumeSpecName: "kube-api-access-tjc8g") pod "9c5adcda-4852-4b4b-bcac-591ca4125c49" (UID: "9c5adcda-4852-4b4b-bcac-591ca4125c49"). InnerVolumeSpecName "kube-api-access-tjc8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:13:39.752728 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.752695    2572 generic.go:356] "Generic (PLEG): container finished" podID="9c5adcda-4852-4b4b-bcac-591ca4125c49" containerID="45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527" exitCode=0
Jul 25 00:13:39.752886 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.752741    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"
Jul 25 00:13:39.752886 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.752776    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b" event={"ID":"9c5adcda-4852-4b4b-bcac-591ca4125c49","Type":"ContainerDied","Data":"45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527"}
Jul 25 00:13:39.752886 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.752830    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b" event={"ID":"9c5adcda-4852-4b4b-bcac-591ca4125c49","Type":"ContainerDied","Data":"2a9ba7328a8fed9c4508aa89fb947d9df1046f3d8e23b57c16f0145bdf2c7c73"}
Jul 25 00:13:39.752886 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.752853    2572 scope.go:117] "RemoveContainer" containerID="45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527"
Jul 25 00:13:39.762334 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.762313    2572 scope.go:117] "RemoveContainer" containerID="45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527"
Jul 25 00:13:39.762679 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:13:39.762661    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527\": container with ID starting with 45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527 not found: ID does not exist" containerID="45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527"
Jul 25 00:13:39.762728 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.762688    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527"} err="failed to get container status \"45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527\": rpc error: code = NotFound desc = could not find container \"45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527\": container with ID starting with 45dcab3b9efcdf7c24362263d4e4205e82c4a8c6c0455afb5ae9a9a4907e5527 not found: ID does not exist"
Jul 25 00:13:39.773951 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.773913    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"]
Jul 25 00:13:39.777761 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.777723    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/auth-test-model-92df89-kserve-5f6d7694f9-plm9b"]
Jul 25 00:13:39.836320 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.836280    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c5adcda-4852-4b4b-bcac-591ca4125c49-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:39.836320 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.836313    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c5adcda-4852-4b4b-bcac-591ca4125c49-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:39.836320 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.836326    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjc8g\" (UniqueName: \"kubernetes.io/projected/9c5adcda-4852-4b4b-bcac-591ca4125c49-kube-api-access-tjc8g\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:39.966814 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:39.966726    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c5adcda-4852-4b4b-bcac-591ca4125c49" path="/var/lib/kubelet/pods/9c5adcda-4852-4b4b-bcac-591ca4125c49/volumes"
Jul 25 00:13:40.206706 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.206660    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25" podUID="a7aae092-85e5-476f-8831-3ad835aafa26" containerName="main" containerID="cri-o://ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99" gracePeriod=45
Jul 25 00:13:40.476972 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.476900    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:13:40.543104 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543070    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-dshm\") pod \"a7aae092-85e5-476f-8831-3ad835aafa26\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") "
Jul 25 00:13:40.543104 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543113    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-tmp-dir\") pod \"a7aae092-85e5-476f-8831-3ad835aafa26\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") "
Jul 25 00:13:40.543370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543155    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7aae092-85e5-476f-8831-3ad835aafa26-tls-certs\") pod \"a7aae092-85e5-476f-8831-3ad835aafa26\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") "
Jul 25 00:13:40.543370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543189    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-home\") pod \"a7aae092-85e5-476f-8831-3ad835aafa26\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") "
Jul 25 00:13:40.543370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543213    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrvth\" (UniqueName: \"kubernetes.io/projected/a7aae092-85e5-476f-8831-3ad835aafa26-kube-api-access-rrvth\") pod \"a7aae092-85e5-476f-8831-3ad835aafa26\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") "
Jul 25 00:13:40.543370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543274    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-model-cache\") pod \"a7aae092-85e5-476f-8831-3ad835aafa26\" (UID: \"a7aae092-85e5-476f-8831-3ad835aafa26\") "
Jul 25 00:13:40.543619 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543499    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7aae092-85e5-476f-8831-3ad835aafa26" (UID: "a7aae092-85e5-476f-8831-3ad835aafa26"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:40.543619 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543522    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-home" (OuterVolumeSpecName: "home") pod "a7aae092-85e5-476f-8831-3ad835aafa26" (UID: "a7aae092-85e5-476f-8831-3ad835aafa26"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:40.543730 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543629    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:40.543730 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543673    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:40.543833 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.543816    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7aae092-85e5-476f-8831-3ad835aafa26" (UID: "a7aae092-85e5-476f-8831-3ad835aafa26"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:40.545637 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.545605    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7aae092-85e5-476f-8831-3ad835aafa26-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7aae092-85e5-476f-8831-3ad835aafa26" (UID: "a7aae092-85e5-476f-8831-3ad835aafa26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:13:40.545637 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.545607    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7aae092-85e5-476f-8831-3ad835aafa26-kube-api-access-rrvth" (OuterVolumeSpecName: "kube-api-access-rrvth") pod "a7aae092-85e5-476f-8831-3ad835aafa26" (UID: "a7aae092-85e5-476f-8831-3ad835aafa26"). InnerVolumeSpecName "kube-api-access-rrvth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:13:40.545834 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.545661    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-dshm" (OuterVolumeSpecName: "dshm") pod "a7aae092-85e5-476f-8831-3ad835aafa26" (UID: "a7aae092-85e5-476f-8831-3ad835aafa26"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:13:40.644473 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.644410    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7aae092-85e5-476f-8831-3ad835aafa26-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:40.644473 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.644490    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrvth\" (UniqueName: \"kubernetes.io/projected/a7aae092-85e5-476f-8831-3ad835aafa26-kube-api-access-rrvth\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:40.644702 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.644502    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:40.644702 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.644510    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7aae092-85e5-476f-8831-3ad835aafa26-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:13:40.760086 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.760047    2572 generic.go:356] "Generic (PLEG): container finished" podID="a7aae092-85e5-476f-8831-3ad835aafa26" containerID="ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99" exitCode=0
Jul 25 00:13:40.760271 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.760096    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"
Jul 25 00:13:40.760271 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.760132    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25" event={"ID":"a7aae092-85e5-476f-8831-3ad835aafa26","Type":"ContainerDied","Data":"ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99"}
Jul 25 00:13:40.760271 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.760179    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25" event={"ID":"a7aae092-85e5-476f-8831-3ad835aafa26","Type":"ContainerDied","Data":"ee719eb69b69c6053fd996c4eca25f7e4103198989c6bf19121b953129b7811c"}
Jul 25 00:13:40.760271 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.760197    2572 scope.go:117] "RemoveContainer" containerID="ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99"
Jul 25 00:13:40.769588 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.769566    2572 scope.go:117] "RemoveContainer" containerID="ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99"
Jul 25 00:13:40.769901 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:13:40.769877    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99\": container with ID starting with ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99 not found: ID does not exist" containerID="ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99"
Jul 25 00:13:40.769958 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.769915    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99"} err="failed to get container status \"ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99\": rpc error: code = NotFound desc = could not find container \"ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99\": container with ID starting with ed322a7fc71395a50e196c4754c18bcf4e5a95f8fcc019a088cf2045b0843a99 not found: ID does not exist"
Jul 25 00:13:40.783354 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.783314    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"]
Jul 25 00:13:40.789712 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:40.789678    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-413844/auth-test-model-413844-kserve-54fc5cc495-7df25"]
Jul 25 00:13:41.966577 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:13:41.966539    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7aae092-85e5-476f-8831-3ad835aafa26" path="/var/lib/kubelet/pods/a7aae092-85e5-476f-8831-3ad835aafa26/volumes"
Jul 25 00:14:00.988378 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.988339    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"]
Jul 25 00:14:00.989110 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.989087    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c5adcda-4852-4b4b-bcac-591ca4125c49" containerName="main"
Jul 25 00:14:00.989110 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.989112    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c5adcda-4852-4b4b-bcac-591ca4125c49" containerName="main"
Jul 25 00:14:00.989278 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.989137    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7aae092-85e5-476f-8831-3ad835aafa26" containerName="main"
Jul 25 00:14:00.989278 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.989147    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7aae092-85e5-476f-8831-3ad835aafa26" containerName="main"
Jul 25 00:14:00.989278 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.989268    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c5adcda-4852-4b4b-bcac-591ca4125c49" containerName="main"
Jul 25 00:14:00.989425 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.989280    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7aae092-85e5-476f-8831-3ad835aafa26" containerName="main"
Jul 25 00:14:00.993972 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.993945    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:00.997290 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.997260    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"openshift-service-ca.crt\""
Jul 25 00:14:00.997518 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.997260    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"default-dockercfg-gzckw\""
Jul 25 00:14:00.997518 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.997261    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"sub-test-model-413844-kserve-self-signed-certs\""
Jul 25 00:14:00.997518 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:00.997319    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-413844\"/\"kube-root-ca.crt\""
Jul 25 00:14:01.001847 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.001816    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"]
Jul 25 00:14:01.149022 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.148979    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-model-cache\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.149202 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.149053    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6ltl\" (UniqueName: \"kubernetes.io/projected/e0fd0556-ae27-4800-bd4c-304d2593f6bd-kube-api-access-w6ltl\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.149202 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.149115    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-dshm\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.149292 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.149202    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tmp-dir\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.149292 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.149250    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-home\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.149387 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.149308    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tls-certs\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.249948 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.249915    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-dshm\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250129 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.249973    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tmp-dir\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250129 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.250019    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-home\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250129 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.250060    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tls-certs\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250129 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.250121    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-model-cache\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250350 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.250166    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6ltl\" (UniqueName: \"kubernetes.io/projected/e0fd0556-ae27-4800-bd4c-304d2593f6bd-kube-api-access-w6ltl\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250481 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.250441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-home\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.250499    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tmp-dir\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.250590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.250531    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-model-cache\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.252289 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.252266    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-dshm\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.252717 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.252697    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tls-certs\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.257311 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.257274    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6ltl\" (UniqueName: \"kubernetes.io/projected/e0fd0556-ae27-4800-bd4c-304d2593f6bd-kube-api-access-w6ltl\") pod \"sub-test-model-413844-kserve-798887f95f-kxmw6\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") " pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.305587 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.305540    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:01.448284 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.448253    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"]
Jul 25 00:14:01.450826 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:14:01.450793    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0fd0556_ae27_4800_bd4c_304d2593f6bd.slice/crio-d5cbfdf4f3e2c2d28165d0ced59b2259e499de05f9cb1ebed46e9e115797e142 WatchSource:0}: Error finding container d5cbfdf4f3e2c2d28165d0ced59b2259e499de05f9cb1ebed46e9e115797e142: Status 404 returned error can't find the container with id d5cbfdf4f3e2c2d28165d0ced59b2259e499de05f9cb1ebed46e9e115797e142
Jul 25 00:14:01.864857 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.864742    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6" event={"ID":"e0fd0556-ae27-4800-bd4c-304d2593f6bd","Type":"ContainerStarted","Data":"2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4"}
Jul 25 00:14:01.864857 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.864796    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6" event={"ID":"e0fd0556-ae27-4800-bd4c-304d2593f6bd","Type":"ContainerStarted","Data":"d5cbfdf4f3e2c2d28165d0ced59b2259e499de05f9cb1ebed46e9e115797e142"}
Jul 25 00:14:01.885660 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:01.885601    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6" podStartSLOduration=1.885582342 podStartE2EDuration="1.885582342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:14:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:14:01.884422787 +0000 UTC m=+2676.380134663" watchObservedRunningTime="2026-07-25 00:14:01.885582342 +0000 UTC m=+2676.381294217"
Jul 25 00:14:04.695678 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:04.695631    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:14:11.306223 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:11.306176    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:11.306856 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:11.306242    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:11.318756 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:11.318723    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:11.925111 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:11.925079    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:14:38.485625 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:38.485588    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:14:56.291884 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:14:56.291849    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:15:00.130509 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.130437    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm"]
Jul 25 00:15:00.134590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.134566    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"]
Jul 25 00:15:00.134693 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.134676    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm"
Jul 25 00:15:00.136871 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.136847    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:15:00.138370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.138349    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"
Jul 25 00:15:00.143200 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.143157    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm"]
Jul 25 00:15:00.145912 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.145883    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"]
Jul 25 00:15:00.149234 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.149142    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm"
Jul 25 00:15:00.156649 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.156629    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"
Jul 25 00:15:00.301140 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.301113    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm"]
Jul 25 00:15:00.303919 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:15:00.303886    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd8e02c_d638_4848_a40d_7af70d08ed7e.slice/crio-cd11c08c33379feecb7c474a21c1334a55ef1913098a5352f25c0cb5c4a977bf WatchSource:0}: Error finding container cd11c08c33379feecb7c474a21c1334a55ef1913098a5352f25c0cb5c4a977bf: Status 404 returned error can't find the container with id cd11c08c33379feecb7c474a21c1334a55ef1913098a5352f25c0cb5c4a977bf
Jul 25 00:15:00.326134 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:00.326103    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"]
Jul 25 00:15:00.328438 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:15:00.328408    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5724e25a_6f89_4dfa_8ef6_af360834ad54.slice/crio-6540dfbf26056571de7c6d4f2cc846fbbb426ac0263d385ef94706aaa9d07aa6 WatchSource:0}: Error finding container 6540dfbf26056571de7c6d4f2cc846fbbb426ac0263d385ef94706aaa9d07aa6: Status 404 returned error can't find the container with id 6540dfbf26056571de7c6d4f2cc846fbbb426ac0263d385ef94706aaa9d07aa6
Jul 25 00:15:01.153163 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:01.153125    2572 generic.go:356] "Generic (PLEG): container finished" podID="5724e25a-6f89-4dfa-8ef6-af360834ad54" containerID="9de81bf31829c71bc1a9e90fd69b49059e5cd1f4ae21213ebcfb772438cbbe9b" exitCode=0
Jul 25 00:15:01.153662 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:01.153207    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf" event={"ID":"5724e25a-6f89-4dfa-8ef6-af360834ad54","Type":"ContainerDied","Data":"9de81bf31829c71bc1a9e90fd69b49059e5cd1f4ae21213ebcfb772438cbbe9b"}
Jul 25 00:15:01.153662 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:01.153295    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf" event={"ID":"5724e25a-6f89-4dfa-8ef6-af360834ad54","Type":"ContainerStarted","Data":"6540dfbf26056571de7c6d4f2cc846fbbb426ac0263d385ef94706aaa9d07aa6"}
Jul 25 00:15:01.154738 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:01.154715    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm" event={"ID":"8cd8e02c-d638-4848-a40d-7af70d08ed7e","Type":"ContainerStarted","Data":"872abe1938aac57faf2f9402b99dd15dc244d7a8a564e78dae5f5c96b983f190"}
Jul 25 00:15:01.154852 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:01.154755    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm" event={"ID":"8cd8e02c-d638-4848-a40d-7af70d08ed7e","Type":"ContainerStarted","Data":"cd11c08c33379feecb7c474a21c1334a55ef1913098a5352f25c0cb5c4a977bf"}
Jul 25 00:15:01.181627 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:01.181577    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm" podStartSLOduration=1.181559477 podStartE2EDuration="1.181559477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:15:01.179977657 +0000 UTC m=+2735.675689534" watchObservedRunningTime="2026-07-25 00:15:01.181559477 +0000 UTC m=+2735.677271353"
Jul 25 00:15:02.161743 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:02.161708    2572 generic.go:356] "Generic (PLEG): container finished" podID="8cd8e02c-d638-4848-a40d-7af70d08ed7e" containerID="872abe1938aac57faf2f9402b99dd15dc244d7a8a564e78dae5f5c96b983f190" exitCode=0
Jul 25 00:15:02.162180 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:02.161791    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm" event={"ID":"8cd8e02c-d638-4848-a40d-7af70d08ed7e","Type":"ContainerDied","Data":"872abe1938aac57faf2f9402b99dd15dc244d7a8a564e78dae5f5c96b983f190"}
Jul 25 00:15:02.302223 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:02.302192    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"
Jul 25 00:15:03.167972 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:03.167939    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"
Jul 25 00:15:03.168395 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:03.167971    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf" event={"ID":"5724e25a-6f89-4dfa-8ef6-af360834ad54","Type":"ContainerDied","Data":"6540dfbf26056571de7c6d4f2cc846fbbb426ac0263d385ef94706aaa9d07aa6"}
Jul 25 00:15:03.168395 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:03.168037    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6540dfbf26056571de7c6d4f2cc846fbbb426ac0263d385ef94706aaa9d07aa6"
Jul 25 00:15:03.309728 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:03.309701    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm"
Jul 25 00:15:04.174981 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:04.174951    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm"
Jul 25 00:15:04.175384 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:04.174989    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748975-7hfxm" event={"ID":"8cd8e02c-d638-4848-a40d-7af70d08ed7e","Type":"ContainerDied","Data":"cd11c08c33379feecb7c474a21c1334a55ef1913098a5352f25c0cb5c4a977bf"}
Jul 25 00:15:04.175384 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:04.175036    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd11c08c33379feecb7c474a21c1334a55ef1913098a5352f25c0cb5c4a977bf"
Jul 25 00:15:10.292054 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:10.292015    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"]
Jul 25 00:15:11.088931 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:11.088890    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:15:24.891033 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.890991    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"]
Jul 25 00:15:24.891658 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.891636    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cd8e02c-d638-4848-a40d-7af70d08ed7e" containerName="cleanup"
Jul 25 00:15:24.891709 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.891664    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cd8e02c-d638-4848-a40d-7af70d08ed7e" containerName="cleanup"
Jul 25 00:15:24.891709 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.891683    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5724e25a-6f89-4dfa-8ef6-af360834ad54" containerName="cleanup"
Jul 25 00:15:24.891709 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.891691    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5724e25a-6f89-4dfa-8ef6-af360834ad54" containerName="cleanup"
Jul 25 00:15:24.891825 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.891813    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cd8e02c-d638-4848-a40d-7af70d08ed7e" containerName="cleanup"
Jul 25 00:15:24.891860 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.891827    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5724e25a-6f89-4dfa-8ef6-af360834ad54" containerName="cleanup"
Jul 25 00:15:24.895130 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.895109    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:24.897606 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.897575    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"openshift-service-ca.crt\""
Jul 25 00:15:24.898491 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.898444    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"kube-root-ca.crt\""
Jul 25 00:15:24.898636 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.898514    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"default-dockercfg-mqhrp\""
Jul 25 00:15:24.898636 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.898588    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-92df89\"/\"rate-test-model-92df89-kserve-self-signed-certs\""
Jul 25 00:15:24.903861 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.903836    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"]
Jul 25 00:15:24.967335 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.967291    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909b07fd-d38f-46e8-bb2e-b02783334377-tls-certs\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:24.967335 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.967332    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-tmp-dir\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:24.967604 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.967351    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts4dq\" (UniqueName: \"kubernetes.io/projected/909b07fd-d38f-46e8-bb2e-b02783334377-kube-api-access-ts4dq\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:24.967604 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.967476    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-home\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:24.967604 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.967512    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-dshm\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:24.967604 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:24.967543    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-model-cache\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.068609 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.068566    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909b07fd-d38f-46e8-bb2e-b02783334377-tls-certs\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.068609 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.068605    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-tmp-dir\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.068873 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.068628    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts4dq\" (UniqueName: \"kubernetes.io/projected/909b07fd-d38f-46e8-bb2e-b02783334377-kube-api-access-ts4dq\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.068873 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.068682    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-home\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.068873 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.068715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-dshm\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.068873 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.068747    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-model-cache\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.069093 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.069060    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-tmp-dir\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.069154 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.069112    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-home\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.069216 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.069195    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-model-cache\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.071129 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.071101    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-dshm\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.071316 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.071296    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909b07fd-d38f-46e8-bb2e-b02783334377-tls-certs\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.075943 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.075918    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts4dq\" (UniqueName: \"kubernetes.io/projected/909b07fd-d38f-46e8-bb2e-b02783334377-kube-api-access-ts4dq\") pod \"rate-test-model-92df89-kserve-6c5df5b96c-bd22b\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") " pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.206515 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.206383    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:25.311773 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.311730    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6" podUID="e0fd0556-ae27-4800-bd4c-304d2593f6bd" containerName="main" containerID="cri-o://2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4" gracePeriod=45
Jul 25 00:15:25.356974 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.356945    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"]
Jul 25 00:15:25.380493 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:15:25.380437    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod909b07fd_d38f_46e8_bb2e_b02783334377.slice/crio-ca6757e95aca0e6653e8fa4be90cafcee5704cf0ca09fc96b448a648de3d3579 WatchSource:0}: Error finding container ca6757e95aca0e6653e8fa4be90cafcee5704cf0ca09fc96b448a648de3d3579: Status 404 returned error can't find the container with id ca6757e95aca0e6653e8fa4be90cafcee5704cf0ca09fc96b448a648de3d3579
Jul 25 00:15:25.391939 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.391905    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:15:25.559846 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.559815    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:15:25.574177 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574141    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6ltl\" (UniqueName: \"kubernetes.io/projected/e0fd0556-ae27-4800-bd4c-304d2593f6bd-kube-api-access-w6ltl\") pod \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") "
Jul 25 00:15:25.574590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574205    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-dshm\") pod \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") "
Jul 25 00:15:25.574590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574242    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tls-certs\") pod \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") "
Jul 25 00:15:25.574590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574272    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-model-cache\") pod \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") "
Jul 25 00:15:25.574590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574299    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tmp-dir\") pod \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") "
Jul 25 00:15:25.574590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574359    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-home\") pod \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\" (UID: \"e0fd0556-ae27-4800-bd4c-304d2593f6bd\") "
Jul 25 00:15:25.574590 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574534    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0fd0556-ae27-4800-bd4c-304d2593f6bd" (UID: "e0fd0556-ae27-4800-bd4c-304d2593f6bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:15:25.574964 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574612    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0fd0556-ae27-4800-bd4c-304d2593f6bd" (UID: "e0fd0556-ae27-4800-bd4c-304d2593f6bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:15:25.574964 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574706    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-home" (OuterVolumeSpecName: "home") pod "e0fd0556-ae27-4800-bd4c-304d2593f6bd" (UID: "e0fd0556-ae27-4800-bd4c-304d2593f6bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:15:25.574964 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574794    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:15:25.574964 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574813    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:15:25.574964 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.574826    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:15:25.577067 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.577034    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-dshm" (OuterVolumeSpecName: "dshm") pod "e0fd0556-ae27-4800-bd4c-304d2593f6bd" (UID: "e0fd0556-ae27-4800-bd4c-304d2593f6bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:15:25.577201 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.577124    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0fd0556-ae27-4800-bd4c-304d2593f6bd" (UID: "e0fd0556-ae27-4800-bd4c-304d2593f6bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:15:25.577201 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.577154    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0fd0556-ae27-4800-bd4c-304d2593f6bd-kube-api-access-w6ltl" (OuterVolumeSpecName: "kube-api-access-w6ltl") pod "e0fd0556-ae27-4800-bd4c-304d2593f6bd" (UID: "e0fd0556-ae27-4800-bd4c-304d2593f6bd"). InnerVolumeSpecName "kube-api-access-w6ltl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:15:25.676079 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.676027    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6ltl\" (UniqueName: \"kubernetes.io/projected/e0fd0556-ae27-4800-bd4c-304d2593f6bd-kube-api-access-w6ltl\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:15:25.676079 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.676069    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0fd0556-ae27-4800-bd4c-304d2593f6bd-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:15:25.676079 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:25.676084    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0fd0556-ae27-4800-bd4c-304d2593f6bd-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:15:26.286051 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.286009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b" event={"ID":"909b07fd-d38f-46e8-bb2e-b02783334377","Type":"ContainerStarted","Data":"f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842"}
Jul 25 00:15:26.286547 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.286065    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b" event={"ID":"909b07fd-d38f-46e8-bb2e-b02783334377","Type":"ContainerStarted","Data":"ca6757e95aca0e6653e8fa4be90cafcee5704cf0ca09fc96b448a648de3d3579"}
Jul 25 00:15:26.287386 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.287354    2572 generic.go:356] "Generic (PLEG): container finished" podID="e0fd0556-ae27-4800-bd4c-304d2593f6bd" containerID="2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4" exitCode=0
Jul 25 00:15:26.287514 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.287416    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"
Jul 25 00:15:26.287514 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.287446    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6" event={"ID":"e0fd0556-ae27-4800-bd4c-304d2593f6bd","Type":"ContainerDied","Data":"2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4"}
Jul 25 00:15:26.287636 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.287510    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6" event={"ID":"e0fd0556-ae27-4800-bd4c-304d2593f6bd","Type":"ContainerDied","Data":"d5cbfdf4f3e2c2d28165d0ced59b2259e499de05f9cb1ebed46e9e115797e142"}
Jul 25 00:15:26.287636 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.287538    2572 scope.go:117] "RemoveContainer" containerID="2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4"
Jul 25 00:15:26.296109 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.296086    2572 scope.go:117] "RemoveContainer" containerID="2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4"
Jul 25 00:15:26.296441 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:15:26.296417    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4\": container with ID starting with 2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4 not found: ID does not exist" containerID="2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4"
Jul 25 00:15:26.296549 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.296471    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4"} err="failed to get container status \"2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4\": rpc error: code = NotFound desc = could not find container \"2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4\": container with ID starting with 2121078057d88f97f59ca5bbcd992f0e803de2e7d160bcb7fbe9cd5933ed6da4 not found: ID does not exist"
Jul 25 00:15:26.303937 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.303880    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b" podStartSLOduration=2.303864271 podStartE2EDuration="2.303864271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:15:26.301286974 +0000 UTC m=+2760.796998848" watchObservedRunningTime="2026-07-25 00:15:26.303864271 +0000 UTC m=+2760.799576145"
Jul 25 00:15:26.314859 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.314817    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"]
Jul 25 00:15:26.318598 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:26.318558    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-413844/sub-test-model-413844-kserve-798887f95f-kxmw6"]
Jul 25 00:15:27.967916 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:27.967881    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0fd0556-ae27-4800-bd4c-304d2593f6bd" path="/var/lib/kubelet/pods/e0fd0556-ae27-4800-bd4c-304d2593f6bd/volumes"
Jul 25 00:15:35.207538 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:35.207488    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:35.207961 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:35.207558    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:35.220175 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:35.220138    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:35.344220 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:35.344191    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:15:36.090683 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:15:36.090644    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:16:11.593095 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:16:11.593055    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:16:54.597743 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:16:54.597706    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:17:16.582566 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:16.582524    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:17:22.596702 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:22.596608    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"]
Jul 25 00:17:24.594493 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.594436    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7561-140074054373424"]
Jul 25 00:17:24.595001 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.594971    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0fd0556-ae27-4800-bd4c-304d2593f6bd" containerName="main"
Jul 25 00:17:24.595001 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.594992    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0fd0556-ae27-4800-bd4c-304d2593f6bd" containerName="main"
Jul 25 00:17:24.595119 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.595106    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0fd0556-ae27-4800-bd4c-304d2593f6bd" containerName="main"
Jul 25 00:17:24.598360 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.598336    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:24.600611 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.600574    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-f4jj2\""
Jul 25 00:17:24.603753 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.603718    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054373424"]
Jul 25 00:17:24.719257 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.719217    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdrdx\" (UniqueName: \"kubernetes.io/projected/11683e2e-199e-457f-a5c5-9e3520b79c3a-kube-api-access-xdrdx\") pod \"test-curl-7561-140074054373424\" (UID: \"11683e2e-199e-457f-a5c5-9e3520b79c3a\") " pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:24.819912 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.819869    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdrdx\" (UniqueName: \"kubernetes.io/projected/11683e2e-199e-457f-a5c5-9e3520b79c3a-kube-api-access-xdrdx\") pod \"test-curl-7561-140074054373424\" (UID: \"11683e2e-199e-457f-a5c5-9e3520b79c3a\") " pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:24.829962 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.829925    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdrdx\" (UniqueName: \"kubernetes.io/projected/11683e2e-199e-457f-a5c5-9e3520b79c3a-kube-api-access-xdrdx\") pod \"test-curl-7561-140074054373424\" (UID: \"11683e2e-199e-457f-a5c5-9e3520b79c3a\") " pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:24.909413 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:24.909321    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:25.050718 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:25.050688    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054373424"]
Jul 25 00:17:25.053639 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:17:25.053612    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11683e2e_199e_457f_a5c5_9e3520b79c3a.slice/crio-8beed8063201033541b66ba0e0e44019c84afddf3732e072ae7dbdc6135683bd WatchSource:0}: Error finding container 8beed8063201033541b66ba0e0e44019c84afddf3732e072ae7dbdc6135683bd: Status 404 returned error can't find the container with id 8beed8063201033541b66ba0e0e44019c84afddf3732e072ae7dbdc6135683bd
Jul 25 00:17:25.054984 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:25.054960    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:17:25.863620 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:25.863577    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054373424" event={"ID":"11683e2e-199e-457f-a5c5-9e3520b79c3a","Type":"ContainerStarted","Data":"8beed8063201033541b66ba0e0e44019c84afddf3732e072ae7dbdc6135683bd"}
Jul 25 00:17:26.871425 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:26.871385    2572 generic.go:356] "Generic (PLEG): container finished" podID="11683e2e-199e-457f-a5c5-9e3520b79c3a" containerID="4ddafb00e8bea8f4c97ddf07a784db469b7d9e7e4bc79f8831c9da301b90f757" exitCode=0
Jul 25 00:17:26.871944 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:26.871484    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054373424" event={"ID":"11683e2e-199e-457f-a5c5-9e3520b79c3a","Type":"ContainerDied","Data":"4ddafb00e8bea8f4c97ddf07a784db469b7d9e7e4bc79f8831c9da301b90f757"}
Jul 25 00:17:28.013440 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.013413    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:28.050923 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.050893    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7561-140074054373424_11683e2e-199e-457f-a5c5-9e3520b79c3a/test-curl-7561-140074054373424/0.log"
Jul 25 00:17:28.053021 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.052993    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdrdx\" (UniqueName: \"kubernetes.io/projected/11683e2e-199e-457f-a5c5-9e3520b79c3a-kube-api-access-xdrdx\") pod \"11683e2e-199e-457f-a5c5-9e3520b79c3a\" (UID: \"11683e2e-199e-457f-a5c5-9e3520b79c3a\") "
Jul 25 00:17:28.055217 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.055183    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11683e2e-199e-457f-a5c5-9e3520b79c3a-kube-api-access-xdrdx" (OuterVolumeSpecName: "kube-api-access-xdrdx") pod "11683e2e-199e-457f-a5c5-9e3520b79c3a" (UID: "11683e2e-199e-457f-a5c5-9e3520b79c3a"). InnerVolumeSpecName "kube-api-access-xdrdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:17:28.075254 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.075216    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7561-140074054373424"]
Jul 25 00:17:28.079050 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.079015    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7561-140074054373424"]
Jul 25 00:17:28.154044 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.153950    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdrdx\" (UniqueName: \"kubernetes.io/projected/11683e2e-199e-457f-a5c5-9e3520b79c3a-kube-api-access-xdrdx\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:28.208004 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.207952    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7561-140074054448304"]
Jul 25 00:17:28.208401 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.208379    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11683e2e-199e-457f-a5c5-9e3520b79c3a" containerName="test-curl-7561-140074054373424"
Jul 25 00:17:28.208401 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.208399    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="11683e2e-199e-457f-a5c5-9e3520b79c3a" containerName="test-curl-7561-140074054373424"
Jul 25 00:17:28.208651 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.208497    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="11683e2e-199e-457f-a5c5-9e3520b79c3a" containerName="test-curl-7561-140074054373424"
Jul 25 00:17:28.211535 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.211510    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:28.217377 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.217347    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054448304"]
Jul 25 00:17:28.255359 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.255316    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp8xb\" (UniqueName: \"kubernetes.io/projected/ff717cef-4992-4879-b67d-cdb4900c8b61-kube-api-access-rp8xb\") pod \"test-curl-7561-140074054448304\" (UID: \"ff717cef-4992-4879-b67d-cdb4900c8b61\") " pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:28.356670 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.356628    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp8xb\" (UniqueName: \"kubernetes.io/projected/ff717cef-4992-4879-b67d-cdb4900c8b61-kube-api-access-rp8xb\") pod \"test-curl-7561-140074054448304\" (UID: \"ff717cef-4992-4879-b67d-cdb4900c8b61\") " pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:28.363691 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.363657    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp8xb\" (UniqueName: \"kubernetes.io/projected/ff717cef-4992-4879-b67d-cdb4900c8b61-kube-api-access-rp8xb\") pod \"test-curl-7561-140074054448304\" (UID: \"ff717cef-4992-4879-b67d-cdb4900c8b61\") " pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:28.522555 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.522524    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:28.663987 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.663877    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054448304"]
Jul 25 00:17:28.666144 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:17:28.666118    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff717cef_4992_4879_b67d_cdb4900c8b61.slice/crio-5564d04d089eac0e079bb269ca6fabe0a576d7572d7212c0e09568ad506aeb13 WatchSource:0}: Error finding container 5564d04d089eac0e079bb269ca6fabe0a576d7572d7212c0e09568ad506aeb13: Status 404 returned error can't find the container with id 5564d04d089eac0e079bb269ca6fabe0a576d7572d7212c0e09568ad506aeb13
Jul 25 00:17:28.882311 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.882216    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054448304" event={"ID":"ff717cef-4992-4879-b67d-cdb4900c8b61","Type":"ContainerStarted","Data":"5564d04d089eac0e079bb269ca6fabe0a576d7572d7212c0e09568ad506aeb13"}
Jul 25 00:17:28.883387 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.883366    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:28.883527 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:28.883369    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8beed8063201033541b66ba0e0e44019c84afddf3732e072ae7dbdc6135683bd"
Jul 25 00:17:28.893844 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:17:28.893805    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7561-140074054373424\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="11683e2e-199e-457f-a5c5-9e3520b79c3a" pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:29.889910 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:29.889871    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff717cef-4992-4879-b67d-cdb4900c8b61" containerID="8e935c5c8f2d1ff27d7731e87a4ae367c4ba578a970da2e4cb925e179640ee6a" exitCode=0
Jul 25 00:17:29.890311 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:29.889957    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054448304" event={"ID":"ff717cef-4992-4879-b67d-cdb4900c8b61","Type":"ContainerDied","Data":"8e935c5c8f2d1ff27d7731e87a4ae367c4ba578a970da2e4cb925e179640ee6a"}
Jul 25 00:17:29.891878 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:17:29.891849    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7561-140074054373424\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="11683e2e-199e-457f-a5c5-9e3520b79c3a" pod="opendatahub/test-curl-7561-140074054373424"
Jul 25 00:17:29.966633 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:29.966595    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11683e2e-199e-457f-a5c5-9e3520b79c3a" path="/var/lib/kubelet/pods/11683e2e-199e-457f-a5c5-9e3520b79c3a/volumes"
Jul 25 00:17:31.031637 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.031606    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:31.050834 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.050804    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7561-140074054448304_ff717cef-4992-4879-b67d-cdb4900c8b61/test-curl-7561-140074054448304/0.log"
Jul 25 00:17:31.074597 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.074559    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7561-140074054448304"]
Jul 25 00:17:31.077329 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.077296    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7561-140074054448304"]
Jul 25 00:17:31.084629 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.084602    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp8xb\" (UniqueName: \"kubernetes.io/projected/ff717cef-4992-4879-b67d-cdb4900c8b61-kube-api-access-rp8xb\") pod \"ff717cef-4992-4879-b67d-cdb4900c8b61\" (UID: \"ff717cef-4992-4879-b67d-cdb4900c8b61\") "
Jul 25 00:17:31.086743 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.086709    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff717cef-4992-4879-b67d-cdb4900c8b61-kube-api-access-rp8xb" (OuterVolumeSpecName: "kube-api-access-rp8xb") pod "ff717cef-4992-4879-b67d-cdb4900c8b61" (UID: "ff717cef-4992-4879-b67d-cdb4900c8b61"). InnerVolumeSpecName "kube-api-access-rp8xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:17:31.185416 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.185317    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp8xb\" (UniqueName: \"kubernetes.io/projected/ff717cef-4992-4879-b67d-cdb4900c8b61-kube-api-access-rp8xb\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:31.249493 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.249438    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:31.249820 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.249807    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff717cef-4992-4879-b67d-cdb4900c8b61" containerName="test-curl-7561-140074054448304"
Jul 25 00:17:31.249881 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.249822    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff717cef-4992-4879-b67d-cdb4900c8b61" containerName="test-curl-7561-140074054448304"
Jul 25 00:17:31.249920 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.249889    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff717cef-4992-4879-b67d-cdb4900c8b61" containerName="test-curl-7561-140074054448304"
Jul 25 00:17:31.252872 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.252851    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:31.258533 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.258500    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:31.286297 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.286235    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58qn\" (UniqueName: \"kubernetes.io/projected/c38f1051-1735-4e55-b955-9d5766be01d9-kube-api-access-c58qn\") pod \"test-curl-7561-140074054440112\" (UID: \"c38f1051-1735-4e55-b955-9d5766be01d9\") " pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:31.387509 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.387438    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58qn\" (UniqueName: \"kubernetes.io/projected/c38f1051-1735-4e55-b955-9d5766be01d9-kube-api-access-c58qn\") pod \"test-curl-7561-140074054440112\" (UID: \"c38f1051-1735-4e55-b955-9d5766be01d9\") " pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:31.394885 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.394860    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58qn\" (UniqueName: \"kubernetes.io/projected/c38f1051-1735-4e55-b955-9d5766be01d9-kube-api-access-c58qn\") pod \"test-curl-7561-140074054440112\" (UID: \"c38f1051-1735-4e55-b955-9d5766be01d9\") " pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:31.562942 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.562905    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:31.699929 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.699885    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:31.702183 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:17:31.702153    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc38f1051_1735_4e55_b955_9d5766be01d9.slice/crio-3be3f72e9c03a11452baad61aab3592fc94446e78ceab82d44315920bd658bc8 WatchSource:0}: Error finding container 3be3f72e9c03a11452baad61aab3592fc94446e78ceab82d44315920bd658bc8: Status 404 returned error can't find the container with id 3be3f72e9c03a11452baad61aab3592fc94446e78ceab82d44315920bd658bc8
Jul 25 00:17:31.901249 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.901157    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:31.901249 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.901208    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5564d04d089eac0e079bb269ca6fabe0a576d7572d7212c0e09568ad506aeb13"
Jul 25 00:17:31.902286 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.902256    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054440112" event={"ID":"c38f1051-1735-4e55-b955-9d5766be01d9","Type":"ContainerStarted","Data":"3be3f72e9c03a11452baad61aab3592fc94446e78ceab82d44315920bd658bc8"}
Jul 25 00:17:31.911871 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:17:31.911836    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7561-140074054448304\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="ff717cef-4992-4879-b67d-cdb4900c8b61" pod="opendatahub/test-curl-7561-140074054448304"
Jul 25 00:17:31.966658 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:31.966616    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff717cef-4992-4879-b67d-cdb4900c8b61" path="/var/lib/kubelet/pods/ff717cef-4992-4879-b67d-cdb4900c8b61/volumes"
Jul 25 00:17:32.908563 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:32.908530    2572 generic.go:356] "Generic (PLEG): container finished" podID="c38f1051-1735-4e55-b955-9d5766be01d9" containerID="333f4e0eb6fe99f794050f0dceda989285dc152c13ac3146242ada63e7fa8e60" exitCode=0
Jul 25 00:17:32.908974 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:32.908592    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054440112" event={"ID":"c38f1051-1735-4e55-b955-9d5766be01d9","Type":"ContainerDied","Data":"333f4e0eb6fe99f794050f0dceda989285dc152c13ac3146242ada63e7fa8e60"}
Jul 25 00:17:34.051149 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.051121    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:34.072073 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.072034    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7561-140074054440112_c38f1051-1735-4e55-b955-9d5766be01d9/test-curl-7561-140074054440112/0.log"
Jul 25 00:17:34.096843 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.096807    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:34.100740 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.100703    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:34.117185 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.117155    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c58qn\" (UniqueName: \"kubernetes.io/projected/c38f1051-1735-4e55-b955-9d5766be01d9-kube-api-access-c58qn\") pod \"c38f1051-1735-4e55-b955-9d5766be01d9\" (UID: \"c38f1051-1735-4e55-b955-9d5766be01d9\") "
Jul 25 00:17:34.119338 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.119304    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c38f1051-1735-4e55-b955-9d5766be01d9-kube-api-access-c58qn" (OuterVolumeSpecName: "kube-api-access-c58qn") pod "c38f1051-1735-4e55-b955-9d5766be01d9" (UID: "c38f1051-1735-4e55-b955-9d5766be01d9"). InnerVolumeSpecName "kube-api-access-c58qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:17:34.217765 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.217681    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:34.218047 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.218034    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c38f1051-1735-4e55-b955-9d5766be01d9" containerName="test-curl-7561-140074054440112"
Jul 25 00:17:34.218099 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.218049    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c38f1051-1735-4e55-b955-9d5766be01d9" containerName="test-curl-7561-140074054440112"
Jul 25 00:17:34.218099 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.218070    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c58qn\" (UniqueName: \"kubernetes.io/projected/c38f1051-1735-4e55-b955-9d5766be01d9-kube-api-access-c58qn\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:34.218174 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.218116    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c38f1051-1735-4e55-b955-9d5766be01d9" containerName="test-curl-7561-140074054440112"
Jul 25 00:17:34.221082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.221061    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:34.224467 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.224423    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:34.319647 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.319600    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnlgw\" (UniqueName: \"kubernetes.io/projected/a0ca4459-5c37-4e86-84cd-e443930261ae-kube-api-access-jnlgw\") pod \"test-curl-7561-140074054440112\" (UID: \"a0ca4459-5c37-4e86-84cd-e443930261ae\") " pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:34.420354 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.420314    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnlgw\" (UniqueName: \"kubernetes.io/projected/a0ca4459-5c37-4e86-84cd-e443930261ae-kube-api-access-jnlgw\") pod \"test-curl-7561-140074054440112\" (UID: \"a0ca4459-5c37-4e86-84cd-e443930261ae\") " pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:34.427546 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.427521    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnlgw\" (UniqueName: \"kubernetes.io/projected/a0ca4459-5c37-4e86-84cd-e443930261ae-kube-api-access-jnlgw\") pod \"test-curl-7561-140074054440112\" (UID: \"a0ca4459-5c37-4e86-84cd-e443930261ae\") " pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:34.532570 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.532527    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:34.674859 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.674830    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:34.677382 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:17:34.677351    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0ca4459_5c37_4e86_84cd_e443930261ae.slice/crio-22b2597bbf90feb5df6310d7b22859771fdaf87ad1367746f43c34031abfb28e WatchSource:0}: Error finding container 22b2597bbf90feb5df6310d7b22859771fdaf87ad1367746f43c34031abfb28e: Status 404 returned error can't find the container with id 22b2597bbf90feb5df6310d7b22859771fdaf87ad1367746f43c34031abfb28e
Jul 25 00:17:34.702559 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.702528    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:17:34.920914 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.920825    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:34.920914 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.920847    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3be3f72e9c03a11452baad61aab3592fc94446e78ceab82d44315920bd658bc8"
Jul 25 00:17:34.922118 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.922091    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054440112" event={"ID":"a0ca4459-5c37-4e86-84cd-e443930261ae","Type":"ContainerStarted","Data":"22b2597bbf90feb5df6310d7b22859771fdaf87ad1367746f43c34031abfb28e"}
Jul 25 00:17:34.932845 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:34.932804    2572 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7561-140074054440112" oldPodUID="c38f1051-1735-4e55-b955-9d5766be01d9" podUID="a0ca4459-5c37-4e86-84cd-e443930261ae"
Jul 25 00:17:35.928470 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:35.928413    2572 generic.go:356] "Generic (PLEG): container finished" podID="a0ca4459-5c37-4e86-84cd-e443930261ae" containerID="ec85023b7f9fcc3d686c7601eddc33b86964d6ba59f227f5416584b5b3989ea3" exitCode=0
Jul 25 00:17:35.928885 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:35.928492    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054440112" event={"ID":"a0ca4459-5c37-4e86-84cd-e443930261ae","Type":"ContainerDied","Data":"ec85023b7f9fcc3d686c7601eddc33b86964d6ba59f227f5416584b5b3989ea3"}
Jul 25 00:17:35.967320 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:35.967283    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c38f1051-1735-4e55-b955-9d5766be01d9" path="/var/lib/kubelet/pods/c38f1051-1735-4e55-b955-9d5766be01d9/volumes"
Jul 25 00:17:37.066148 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.066118    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:37.084282 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.084254    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7561-140074054440112_a0ca4459-5c37-4e86-84cd-e443930261ae/test-curl-7561-140074054440112/0.log"
Jul 25 00:17:37.106911 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.106832    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:37.109518 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.109485    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7561-140074054440112"]
Jul 25 00:17:37.146172 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.146138    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnlgw\" (UniqueName: \"kubernetes.io/projected/a0ca4459-5c37-4e86-84cd-e443930261ae-kube-api-access-jnlgw\") pod \"a0ca4459-5c37-4e86-84cd-e443930261ae\" (UID: \"a0ca4459-5c37-4e86-84cd-e443930261ae\") "
Jul 25 00:17:37.148319 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.148286    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0ca4459-5c37-4e86-84cd-e443930261ae-kube-api-access-jnlgw" (OuterVolumeSpecName: "kube-api-access-jnlgw") pod "a0ca4459-5c37-4e86-84cd-e443930261ae" (UID: "a0ca4459-5c37-4e86-84cd-e443930261ae"). InnerVolumeSpecName "kube-api-access-jnlgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:17:37.237882 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.237783    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:37.238310 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.238289    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ca4459-5c37-4e86-84cd-e443930261ae" containerName="test-curl-7561-140074054440112"
Jul 25 00:17:37.238418 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.238323    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ca4459-5c37-4e86-84cd-e443930261ae" containerName="test-curl-7561-140074054440112"
Jul 25 00:17:37.238418 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.238411    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ca4459-5c37-4e86-84cd-e443930261ae" containerName="test-curl-7561-140074054440112"
Jul 25 00:17:37.241447 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.241413    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:37.244467 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.244434    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:37.246930 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.246907    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnlgw\" (UniqueName: \"kubernetes.io/projected/a0ca4459-5c37-4e86-84cd-e443930261ae-kube-api-access-jnlgw\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:37.347822 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.347775    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgjtw\" (UniqueName: \"kubernetes.io/projected/cb496ac8-7b2f-4a28-a925-23ef3289bbac-kube-api-access-pgjtw\") pod \"test-curl-7561-140074054441120\" (UID: \"cb496ac8-7b2f-4a28-a925-23ef3289bbac\") " pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:37.448921 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.448860    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgjtw\" (UniqueName: \"kubernetes.io/projected/cb496ac8-7b2f-4a28-a925-23ef3289bbac-kube-api-access-pgjtw\") pod \"test-curl-7561-140074054441120\" (UID: \"cb496ac8-7b2f-4a28-a925-23ef3289bbac\") " pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:37.456558 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.456530    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgjtw\" (UniqueName: \"kubernetes.io/projected/cb496ac8-7b2f-4a28-a925-23ef3289bbac-kube-api-access-pgjtw\") pod \"test-curl-7561-140074054441120\" (UID: \"cb496ac8-7b2f-4a28-a925-23ef3289bbac\") " pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:37.551879 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.551834    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:37.616161 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.616122    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b" podUID="909b07fd-d38f-46e8-bb2e-b02783334377" containerName="main" containerID="cri-o://f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842" gracePeriod=45
Jul 25 00:17:37.694377 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.694348    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:37.696846 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:17:37.696818    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb496ac8_7b2f_4a28_a925_23ef3289bbac.slice/crio-fb7ad76187bdf2e86f75d1b2663e828a763e0b740dc520fdc17588a928a37125 WatchSource:0}: Error finding container fb7ad76187bdf2e86f75d1b2663e828a763e0b740dc520fdc17588a928a37125: Status 404 returned error can't find the container with id fb7ad76187bdf2e86f75d1b2663e828a763e0b740dc520fdc17588a928a37125
Jul 25 00:17:37.857373 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.857343    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:17:37.940282 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.940250    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:37.940282 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.940275    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22b2597bbf90feb5df6310d7b22859771fdaf87ad1367746f43c34031abfb28e"
Jul 25 00:17:37.941504 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.941474    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054441120" event={"ID":"cb496ac8-7b2f-4a28-a925-23ef3289bbac","Type":"ContainerStarted","Data":"fb7ad76187bdf2e86f75d1b2663e828a763e0b740dc520fdc17588a928a37125"}
Jul 25 00:17:37.942686 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.942656    2572 generic.go:356] "Generic (PLEG): container finished" podID="909b07fd-d38f-46e8-bb2e-b02783334377" containerID="f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842" exitCode=0
Jul 25 00:17:37.942805 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.942713    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"
Jul 25 00:17:37.942805 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.942737    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b" event={"ID":"909b07fd-d38f-46e8-bb2e-b02783334377","Type":"ContainerDied","Data":"f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842"}
Jul 25 00:17:37.942805 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.942787    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b" event={"ID":"909b07fd-d38f-46e8-bb2e-b02783334377","Type":"ContainerDied","Data":"ca6757e95aca0e6653e8fa4be90cafcee5704cf0ca09fc96b448a648de3d3579"}
Jul 25 00:17:37.942976 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.942811    2572 scope.go:117] "RemoveContainer" containerID="f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842"
Jul 25 00:17:37.951116 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:17:37.951079    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7561-140074054440112\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="a0ca4459-5c37-4e86-84cd-e443930261ae" pod="opendatahub/test-curl-7561-140074054440112"
Jul 25 00:17:37.952207 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952187    2572 scope.go:117] "RemoveContainer" containerID="f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842"
Jul 25 00:17:37.952553 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:17:37.952536    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842\": container with ID starting with f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842 not found: ID does not exist" containerID="f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842"
Jul 25 00:17:37.952614 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952562    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842"} err="failed to get container status \"f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842\": rpc error: code = NotFound desc = could not find container \"f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842\": container with ID starting with f1f902750d24fdb6a995472a6772f51de8d9240fd4164097ec72fc511e9c5842 not found: ID does not exist"
Jul 25 00:17:37.952750 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952734    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-dshm\") pod \"909b07fd-d38f-46e8-bb2e-b02783334377\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") "
Jul 25 00:17:37.952789 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952772    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-tmp-dir\") pod \"909b07fd-d38f-46e8-bb2e-b02783334377\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") "
Jul 25 00:17:37.952826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952801    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-home\") pod \"909b07fd-d38f-46e8-bb2e-b02783334377\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") "
Jul 25 00:17:37.952903 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952888    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-model-cache\") pod \"909b07fd-d38f-46e8-bb2e-b02783334377\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") "
Jul 25 00:17:37.952954 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952940    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts4dq\" (UniqueName: \"kubernetes.io/projected/909b07fd-d38f-46e8-bb2e-b02783334377-kube-api-access-ts4dq\") pod \"909b07fd-d38f-46e8-bb2e-b02783334377\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") "
Jul 25 00:17:37.953012 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.952979    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909b07fd-d38f-46e8-bb2e-b02783334377-tls-certs\") pod \"909b07fd-d38f-46e8-bb2e-b02783334377\" (UID: \"909b07fd-d38f-46e8-bb2e-b02783334377\") "
Jul 25 00:17:37.953064 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.953043    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-home" (OuterVolumeSpecName: "home") pod "909b07fd-d38f-46e8-bb2e-b02783334377" (UID: "909b07fd-d38f-46e8-bb2e-b02783334377"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:17:37.953179 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.953158    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-model-cache" (OuterVolumeSpecName: "model-cache") pod "909b07fd-d38f-46e8-bb2e-b02783334377" (UID: "909b07fd-d38f-46e8-bb2e-b02783334377"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:17:37.953269 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.953249    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "909b07fd-d38f-46e8-bb2e-b02783334377" (UID: "909b07fd-d38f-46e8-bb2e-b02783334377"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:17:37.953338 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.953320    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:37.953388 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.953345    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:37.954972 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.954952    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-dshm" (OuterVolumeSpecName: "dshm") pod "909b07fd-d38f-46e8-bb2e-b02783334377" (UID: "909b07fd-d38f-46e8-bb2e-b02783334377"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:17:37.955047 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.955013    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/909b07fd-d38f-46e8-bb2e-b02783334377-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "909b07fd-d38f-46e8-bb2e-b02783334377" (UID: "909b07fd-d38f-46e8-bb2e-b02783334377"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:17:37.955081 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.955054    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909b07fd-d38f-46e8-bb2e-b02783334377-kube-api-access-ts4dq" (OuterVolumeSpecName: "kube-api-access-ts4dq") pod "909b07fd-d38f-46e8-bb2e-b02783334377" (UID: "909b07fd-d38f-46e8-bb2e-b02783334377"). InnerVolumeSpecName "kube-api-access-ts4dq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:17:37.968346 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:37.968313    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0ca4459-5c37-4e86-84cd-e443930261ae" path="/var/lib/kubelet/pods/a0ca4459-5c37-4e86-84cd-e443930261ae/volumes"
Jul 25 00:17:38.054427 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.054397    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts4dq\" (UniqueName: \"kubernetes.io/projected/909b07fd-d38f-46e8-bb2e-b02783334377-kube-api-access-ts4dq\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:38.054603 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.054431    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/909b07fd-d38f-46e8-bb2e-b02783334377-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:38.054603 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.054448    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:38.054603 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.054487    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909b07fd-d38f-46e8-bb2e-b02783334377-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:38.257372 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.257335    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"]
Jul 25 00:17:38.261741 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.261707    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-92df89/rate-test-model-92df89-kserve-6c5df5b96c-bd22b"]
Jul 25 00:17:38.950233 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.950197    2572 generic.go:356] "Generic (PLEG): container finished" podID="cb496ac8-7b2f-4a28-a925-23ef3289bbac" containerID="a0ae15a965c6a19969234cbc37d55b22c2d5f8676a829a093b01e9ba14827ac6" exitCode=0
Jul 25 00:17:38.950439 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:38.950279    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054441120" event={"ID":"cb496ac8-7b2f-4a28-a925-23ef3289bbac","Type":"ContainerDied","Data":"a0ae15a965c6a19969234cbc37d55b22c2d5f8676a829a093b01e9ba14827ac6"}
Jul 25 00:17:39.967407 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:39.967373    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="909b07fd-d38f-46e8-bb2e-b02783334377" path="/var/lib/kubelet/pods/909b07fd-d38f-46e8-bb2e-b02783334377/volumes"
Jul 25 00:17:40.108218 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.108191    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:40.126994 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.126951    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7561-140074054441120_cb496ac8-7b2f-4a28-a925-23ef3289bbac/test-curl-7561-140074054441120/0.log"
Jul 25 00:17:40.151902 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.151863    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:40.155546 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.155510    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:40.175978 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.175933    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgjtw\" (UniqueName: \"kubernetes.io/projected/cb496ac8-7b2f-4a28-a925-23ef3289bbac-kube-api-access-pgjtw\") pod \"cb496ac8-7b2f-4a28-a925-23ef3289bbac\" (UID: \"cb496ac8-7b2f-4a28-a925-23ef3289bbac\") "
Jul 25 00:17:40.178259 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.178222    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb496ac8-7b2f-4a28-a925-23ef3289bbac-kube-api-access-pgjtw" (OuterVolumeSpecName: "kube-api-access-pgjtw") pod "cb496ac8-7b2f-4a28-a925-23ef3289bbac" (UID: "cb496ac8-7b2f-4a28-a925-23ef3289bbac"). InnerVolumeSpecName "kube-api-access-pgjtw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:17:40.276808 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.276773    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgjtw\" (UniqueName: \"kubernetes.io/projected/cb496ac8-7b2f-4a28-a925-23ef3289bbac-kube-api-access-pgjtw\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:40.281964 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.281931    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:40.282289 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.282275    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb496ac8-7b2f-4a28-a925-23ef3289bbac" containerName="test-curl-7561-140074054441120"
Jul 25 00:17:40.282340 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.282290    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb496ac8-7b2f-4a28-a925-23ef3289bbac" containerName="test-curl-7561-140074054441120"
Jul 25 00:17:40.282340 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.282310    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="909b07fd-d38f-46e8-bb2e-b02783334377" containerName="main"
Jul 25 00:17:40.282340 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.282316    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="909b07fd-d38f-46e8-bb2e-b02783334377" containerName="main"
Jul 25 00:17:40.282441 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.282368    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="909b07fd-d38f-46e8-bb2e-b02783334377" containerName="main"
Jul 25 00:17:40.282441 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.282379    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb496ac8-7b2f-4a28-a925-23ef3289bbac" containerName="test-curl-7561-140074054441120"
Jul 25 00:17:40.285365 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.285345    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:40.291415 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.291362    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:40.378039 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.377998    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxzgl\" (UniqueName: \"kubernetes.io/projected/5f81deaa-0f8b-48ba-bc84-b8420fa366b8-kube-api-access-pxzgl\") pod \"test-curl-7561-140074054441120\" (UID: \"5f81deaa-0f8b-48ba-bc84-b8420fa366b8\") " pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:40.479377 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.479329    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxzgl\" (UniqueName: \"kubernetes.io/projected/5f81deaa-0f8b-48ba-bc84-b8420fa366b8-kube-api-access-pxzgl\") pod \"test-curl-7561-140074054441120\" (UID: \"5f81deaa-0f8b-48ba-bc84-b8420fa366b8\") " pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:40.486780 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.486751    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxzgl\" (UniqueName: \"kubernetes.io/projected/5f81deaa-0f8b-48ba-bc84-b8420fa366b8-kube-api-access-pxzgl\") pod \"test-curl-7561-140074054441120\" (UID: \"5f81deaa-0f8b-48ba-bc84-b8420fa366b8\") " pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:40.596064 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.595951    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:40.735270 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.735218    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:40.737572 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:17:40.737542    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f81deaa_0f8b_48ba_bc84_b8420fa366b8.slice/crio-d472f1d89324d9d441bf9ecc8a4cc012b3b5b1d37345c8a1c376e7a3eb88486b WatchSource:0}: Error finding container d472f1d89324d9d441bf9ecc8a4cc012b3b5b1d37345c8a1c376e7a3eb88486b: Status 404 returned error can't find the container with id d472f1d89324d9d441bf9ecc8a4cc012b3b5b1d37345c8a1c376e7a3eb88486b
Jul 25 00:17:40.962171 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.962065    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054441120" event={"ID":"5f81deaa-0f8b-48ba-bc84-b8420fa366b8","Type":"ContainerStarted","Data":"d472f1d89324d9d441bf9ecc8a4cc012b3b5b1d37345c8a1c376e7a3eb88486b"}
Jul 25 00:17:40.963370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.963351    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:40.963475 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.963390    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb7ad76187bdf2e86f75d1b2663e828a763e0b740dc520fdc17588a928a37125"
Jul 25 00:17:40.977185 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:40.977149    2572 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7561-140074054441120" oldPodUID="cb496ac8-7b2f-4a28-a925-23ef3289bbac" podUID="5f81deaa-0f8b-48ba-bc84-b8420fa366b8"
Jul 25 00:17:41.967218 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:41.967182    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb496ac8-7b2f-4a28-a925-23ef3289bbac" path="/var/lib/kubelet/pods/cb496ac8-7b2f-4a28-a925-23ef3289bbac/volumes"
Jul 25 00:17:41.969351 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:41.969327    2572 generic.go:356] "Generic (PLEG): container finished" podID="5f81deaa-0f8b-48ba-bc84-b8420fa366b8" containerID="3c062136aa84327e99b28cbb26299817331b5780afc19ba9a1f1e0581578c678" exitCode=0
Jul 25 00:17:41.969495 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:41.969415    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7561-140074054441120" event={"ID":"5f81deaa-0f8b-48ba-bc84-b8420fa366b8","Type":"ContainerDied","Data":"3c062136aa84327e99b28cbb26299817331b5780afc19ba9a1f1e0581578c678"}
Jul 25 00:17:43.115097 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.115070    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:43.133696 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.133662    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7561-140074054441120_5f81deaa-0f8b-48ba-bc84-b8420fa366b8/test-curl-7561-140074054441120/0.log"
Jul 25 00:17:43.158402 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.158361    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:43.162196 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.162165    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7561-140074054441120"]
Jul 25 00:17:43.203961 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.203930    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxzgl\" (UniqueName: \"kubernetes.io/projected/5f81deaa-0f8b-48ba-bc84-b8420fa366b8-kube-api-access-pxzgl\") pod \"5f81deaa-0f8b-48ba-bc84-b8420fa366b8\" (UID: \"5f81deaa-0f8b-48ba-bc84-b8420fa366b8\") "
Jul 25 00:17:43.206116 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.206084    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f81deaa-0f8b-48ba-bc84-b8420fa366b8-kube-api-access-pxzgl" (OuterVolumeSpecName: "kube-api-access-pxzgl") pod "5f81deaa-0f8b-48ba-bc84-b8420fa366b8" (UID: "5f81deaa-0f8b-48ba-bc84-b8420fa366b8"). InnerVolumeSpecName "kube-api-access-pxzgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:17:43.305435 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.305391    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxzgl\" (UniqueName: \"kubernetes.io/projected/5f81deaa-0f8b-48ba-bc84-b8420fa366b8-kube-api-access-pxzgl\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:17:43.967081 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.967046    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f81deaa-0f8b-48ba-bc84-b8420fa366b8" path="/var/lib/kubelet/pods/5f81deaa-0f8b-48ba-bc84-b8420fa366b8/volumes"
Jul 25 00:17:43.981392 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.981365    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7561-140074054441120"
Jul 25 00:17:43.981607 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:43.981408    2572 scope.go:117] "RemoveContainer" containerID="3c062136aa84327e99b28cbb26299817331b5780afc19ba9a1f1e0581578c678"
Jul 25 00:17:56.092715 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:17:56.092668    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:18:02.712110 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.712068    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"]
Jul 25 00:18:02.712599 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.712575    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f81deaa-0f8b-48ba-bc84-b8420fa366b8" containerName="test-curl-7561-140074054441120"
Jul 25 00:18:02.712599 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.712592    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f81deaa-0f8b-48ba-bc84-b8420fa366b8" containerName="test-curl-7561-140074054441120"
Jul 25 00:18:02.712794 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.712651    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f81deaa-0f8b-48ba-bc84-b8420fa366b8" containerName="test-curl-7561-140074054441120"
Jul 25 00:18:02.718187 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.718156    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.720668 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.720643    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-899559-openshift-default-dockercfg-bglfb\""
Jul 25 00:18:02.726918 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.726888    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"]
Jul 25 00:18:02.795719 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795676    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-certs\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.795719 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795722    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-podinfo\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.795934 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795747    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-data\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.795934 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795774    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-envoy\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.795934 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795808    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istiod-ca-cert\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.795934 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795833    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-credential-socket\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.795934 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795860    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72fkl\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-kube-api-access-72fkl\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.795934 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795891    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-socket\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.796147 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.795962    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-token\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897228 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897184    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-socket\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897228 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897246    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-token\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897286    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-certs\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897302    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-podinfo\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897324    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-data\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897345    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-envoy\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897388    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istiod-ca-cert\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897419    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-credential-socket\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897478    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72fkl\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-kube-api-access-72fkl\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897906 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897729    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-socket\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897906 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-data\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.897906 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897871    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-credential-socket\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.898068 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.897957    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-certs\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.898270 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.898246    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istiod-ca-cert\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.899902 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.899873    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-envoy\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.900049 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.900030    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-podinfo\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.905185 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.905148    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-token\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:02.905349 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:02.905329    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72fkl\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-kube-api-access-72fkl\") pod \"e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") " pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:03.030917 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:03.030873    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:03.180094 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:03.180069    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"]
Jul 25 00:18:03.182230 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:18:03.182202    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb185eeb5_f2b2_4fe8_8c66_9413c963ea52.slice/crio-35ee34d3790f2007743bea800492fe4c3979c48e2d21a22d8a9ab65bcd5c4156 WatchSource:0}: Error finding container 35ee34d3790f2007743bea800492fe4c3979c48e2d21a22d8a9ab65bcd5c4156: Status 404 returned error can't find the container with id 35ee34d3790f2007743bea800492fe4c3979c48e2d21a22d8a9ab65bcd5c4156
Jul 25 00:18:03.184592 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:03.184560    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:18:03.184692 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:03.184628    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:18:03.184692 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:03.184657    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:18:04.088651 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:04.088612    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" event={"ID":"b185eeb5-f2b2-4fe8-8c66-9413c963ea52","Type":"ContainerStarted","Data":"a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2"}
Jul 25 00:18:04.089076 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:04.088678    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" event={"ID":"b185eeb5-f2b2-4fe8-8c66-9413c963ea52","Type":"ContainerStarted","Data":"35ee34d3790f2007743bea800492fe4c3979c48e2d21a22d8a9ab65bcd5c4156"}
Jul 25 00:18:04.108927 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:04.108872    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" podStartSLOduration=2.108852424 podStartE2EDuration="2.108852424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:18:04.107624427 +0000 UTC m=+2918.603336304" watchObservedRunningTime="2026-07-25 00:18:04.108852424 +0000 UTC m=+2918.604564300"
Jul 25 00:18:05.031518 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:05.031472    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:05.036880 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:05.036845    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:05.093780 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:05.093754    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:05.094726 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:05.094703    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:14.191923 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:14.191884    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:18:21.135643 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:21.135596    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 25 00:18:21.136162 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:21.135651    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 25 00:18:23.188396 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:23.188360    2572 generic.go:356] "Generic (PLEG): container finished" podID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerID="a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2" exitCode=0
Jul 25 00:18:23.188866 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:23.188423    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" event={"ID":"b185eeb5-f2b2-4fe8-8c66-9413c963ea52","Type":"ContainerDied","Data":"a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2"}
Jul 25 00:18:23.188916 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:23.188896    2572 scope.go:117] "RemoveContainer" containerID="a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2"
Jul 25 00:18:23.190097 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:23.190054    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:18:23.190189 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:23.190134    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:18:23.190189 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:23.190172    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 25 00:18:24.199507 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:24.199434    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" event={"ID":"b185eeb5-f2b2-4fe8-8c66-9413c963ea52","Type":"ContainerStarted","Data":"84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea"}
Jul 25 00:18:25.031886 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:25.031840    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:25.032184 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:25.032156    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 25 00:18:25.032254 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:25.032209    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 25 00:18:26.032090 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:26.032049    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 25 00:18:26.032548 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:26.032116    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 25 00:18:27.034015 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:27.033969    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 25 00:18:27.034528 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:27.034050    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 25 00:18:28.032112 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:28.032067    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 25 00:18:28.032294 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:28.032129    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 25 00:18:29.035911 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:29.035873    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:29.036398 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:29.036186    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:29.037090 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:29.037073    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:18:33.480287 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.480244    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"]
Jul 25 00:18:33.485676 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.485648    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.489258 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.489018    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-be7457\"/\"openshift-service-ca.crt\""
Jul 25 00:18:33.489258 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.489018    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-be7457\"/\"default-dockercfg-hbx8x\""
Jul 25 00:18:33.489258 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.489018    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-be7457\"/\"ipp-route-be7457-kserve-self-signed-certs\""
Jul 25 00:18:33.489258 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.489028    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-be7457\"/\"kube-root-ca.crt\""
Jul 25 00:18:33.492470 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.492427    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"]
Jul 25 00:18:33.599936 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.599896    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/146094f8-c102-4935-9c33-3201859a59de-tls-certs\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.600128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.599956    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-model-cache\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.600128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.599988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-home\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.600128 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.600015    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-tmp-dir\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.600241 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.600124    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-dshm\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.600241 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.600227    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zd8z\" (UniqueName: \"kubernetes.io/projected/146094f8-c102-4935-9c33-3201859a59de-kube-api-access-4zd8z\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.702295 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.702241    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/146094f8-c102-4935-9c33-3201859a59de-tls-certs\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.702543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.702332    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-model-cache\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.702543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.702373    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-home\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.702543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.702405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-tmp-dir\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.702543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.702476    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-dshm\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.702752 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.702579    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zd8z\" (UniqueName: \"kubernetes.io/projected/146094f8-c102-4935-9c33-3201859a59de-kube-api-access-4zd8z\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.703720 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.703688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-tmp-dir\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.704108 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.704083    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-model-cache\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.704370 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.704351    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-home\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.709086 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.707725    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-dshm\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.709993 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.709970    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/146094f8-c102-4935-9c33-3201859a59de-tls-certs\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.710772 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.710750    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zd8z\" (UniqueName: \"kubernetes.io/projected/146094f8-c102-4935-9c33-3201859a59de-kube-api-access-4zd8z\") pod \"ipp-route-be7457-kserve-5f84c64f4c-gmxn2\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") " pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.798484 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.798410    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:33.941381 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:33.941354    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"]
Jul 25 00:18:33.943549 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:18:33.943521    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod146094f8_c102_4935_9c33_3201859a59de.slice/crio-2e343776e15e620556bc1774e3b896f8e73d50d69c72d97794a677fe9c782790 WatchSource:0}: Error finding container 2e343776e15e620556bc1774e3b896f8e73d50d69c72d97794a677fe9c782790: Status 404 returned error can't find the container with id 2e343776e15e620556bc1774e3b896f8e73d50d69c72d97794a677fe9c782790
Jul 25 00:18:34.254036 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:34.254000    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2" event={"ID":"146094f8-c102-4935-9c33-3201859a59de","Type":"ContainerStarted","Data":"2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79"}
Jul 25 00:18:34.254222 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:34.254046    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2" event={"ID":"146094f8-c102-4935-9c33-3201859a59de","Type":"ContainerStarted","Data":"2e343776e15e620556bc1774e3b896f8e73d50d69c72d97794a677fe9c782790"}
Jul 25 00:18:34.270577 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:34.270514    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2" podStartSLOduration=1.270497711 podStartE2EDuration="1.270497711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:18:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:18:34.268833095 +0000 UTC m=+2948.764544971" watchObservedRunningTime="2026-07-25 00:18:34.270497711 +0000 UTC m=+2948.766209630"
Jul 25 00:18:37.789204 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:37.789156    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:18:43.798718 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:43.798674    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:43.798718 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:43.798726    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:43.811617 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:43.811582    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:18:44.314634 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:18:44.314599    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:19:03.085339 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:03.085302    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:19:29.716637 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:29.716594    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:19:39.138796 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.138747    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"]
Jul 25 00:19:39.142532 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.142496    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"
Jul 25 00:19:39.144714 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.144676    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:19:39.144714 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.144712    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 00:19:39.149684 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.149651    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"]
Jul 25 00:19:39.252432 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.252387    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e06bac-c805-4e6e-ba2e-939a85ae05a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q\" (UID: \"71e06bac-c805-4e6e-ba2e-939a85ae05a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"
Jul 25 00:19:39.353814 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.353767    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e06bac-c805-4e6e-ba2e-939a85ae05a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q\" (UID: \"71e06bac-c805-4e6e-ba2e-939a85ae05a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"
Jul 25 00:19:39.354504 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.354443    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e06bac-c805-4e6e-ba2e-939a85ae05a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q\" (UID: \"71e06bac-c805-4e6e-ba2e-939a85ae05a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"
Jul 25 00:19:39.456987 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.456890    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"
Jul 25 00:19:39.601049 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:39.601019    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"]
Jul 25 00:19:39.603298 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:19:39.603263    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e06bac_c805_4e6e_ba2e_939a85ae05a3.slice/crio-96320566e561d82f1c12f8a972f74f2c96c4a9bdf75609ee3cbd1ce1b5321aff WatchSource:0}: Error finding container 96320566e561d82f1c12f8a972f74f2c96c4a9bdf75609ee3cbd1ce1b5321aff: Status 404 returned error can't find the container with id 96320566e561d82f1c12f8a972f74f2c96c4a9bdf75609ee3cbd1ce1b5321aff
Jul 25 00:19:40.586939 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:40.586835    2572 generic.go:356] "Generic (PLEG): container finished" podID="71e06bac-c805-4e6e-ba2e-939a85ae05a3" containerID="a2e57ea6b6672c0c38b5018e19a969486c89c477345613a4d5377da0f8c0d4cb" exitCode=0
Jul 25 00:19:40.586939 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:40.586904    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q" event={"ID":"71e06bac-c805-4e6e-ba2e-939a85ae05a3","Type":"ContainerDied","Data":"a2e57ea6b6672c0c38b5018e19a969486c89c477345613a4d5377da0f8c0d4cb"}
Jul 25 00:19:40.587334 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:40.586954    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q" event={"ID":"71e06bac-c805-4e6e-ba2e-939a85ae05a3","Type":"ContainerStarted","Data":"96320566e561d82f1c12f8a972f74f2c96c4a9bdf75609ee3cbd1ce1b5321aff"}
Jul 25 00:19:41.734704 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:41.734674    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"
Jul 25 00:19:41.880187 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:41.880091    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e06bac-c805-4e6e-ba2e-939a85ae05a3-maas-api-service-ca\") pod \"71e06bac-c805-4e6e-ba2e-939a85ae05a3\" (UID: \"71e06bac-c805-4e6e-ba2e-939a85ae05a3\") "
Jul 25 00:19:41.880513 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:41.880490    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e06bac-c805-4e6e-ba2e-939a85ae05a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71e06bac-c805-4e6e-ba2e-939a85ae05a3" (UID: "71e06bac-c805-4e6e-ba2e-939a85ae05a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:19:41.980889 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:41.980860    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71e06bac-c805-4e6e-ba2e-939a85ae05a3-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:19:42.599223 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:42.599190    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"
Jul 25 00:19:42.599223 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:42.599199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q" event={"ID":"71e06bac-c805-4e6e-ba2e-939a85ae05a3","Type":"ContainerDied","Data":"96320566e561d82f1c12f8a972f74f2c96c4a9bdf75609ee3cbd1ce1b5321aff"}
Jul 25 00:19:42.599449 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:42.599246    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96320566e561d82f1c12f8a972f74f2c96c4a9bdf75609ee3cbd1ce1b5321aff"
Jul 25 00:19:54.211742 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:54.211704    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"]
Jul 25 00:19:54.214306 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:54.214277    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-85930d-4hx7q"]
Jul 25 00:19:55.967026 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:55.966990    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e06bac-c805-4e6e-ba2e-939a85ae05a3" path="/var/lib/kubelet/pods/71e06bac-c805-4e6e-ba2e-939a85ae05a3/volumes"
Jul 25 00:19:56.591646 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:19:56.591606    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:20:20.494087 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:20.493990    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:20:30.294539 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:30.294495    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"]
Jul 25 00:20:35.385013 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:35.384973    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:20:45.314493 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.314400    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2" podUID="146094f8-c102-4935-9c33-3201859a59de" containerName="main" containerID="cri-o://2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79" gracePeriod=45
Jul 25 00:20:45.594047 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.594017    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:20:45.715034 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.714997    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/146094f8-c102-4935-9c33-3201859a59de-tls-certs\") pod \"146094f8-c102-4935-9c33-3201859a59de\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") "
Jul 25 00:20:45.715227 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715049    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-model-cache\") pod \"146094f8-c102-4935-9c33-3201859a59de\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") "
Jul 25 00:20:45.715227 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715114    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-home\") pod \"146094f8-c102-4935-9c33-3201859a59de\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") "
Jul 25 00:20:45.715227 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715159    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zd8z\" (UniqueName: \"kubernetes.io/projected/146094f8-c102-4935-9c33-3201859a59de-kube-api-access-4zd8z\") pod \"146094f8-c102-4935-9c33-3201859a59de\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") "
Jul 25 00:20:45.715227 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715222    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-dshm\") pod \"146094f8-c102-4935-9c33-3201859a59de\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") "
Jul 25 00:20:45.715488 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715255    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-tmp-dir\") pod \"146094f8-c102-4935-9c33-3201859a59de\" (UID: \"146094f8-c102-4935-9c33-3201859a59de\") "
Jul 25 00:20:45.715488 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715374    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-model-cache" (OuterVolumeSpecName: "model-cache") pod "146094f8-c102-4935-9c33-3201859a59de" (UID: "146094f8-c102-4935-9c33-3201859a59de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:20:45.715607 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715500    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-home" (OuterVolumeSpecName: "home") pod "146094f8-c102-4935-9c33-3201859a59de" (UID: "146094f8-c102-4935-9c33-3201859a59de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:20:45.715682 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715644    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "146094f8-c102-4935-9c33-3201859a59de" (UID: "146094f8-c102-4935-9c33-3201859a59de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:20:45.715682 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715667    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:20:45.715783 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.715687    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:20:45.718045 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.717935    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/146094f8-c102-4935-9c33-3201859a59de-kube-api-access-4zd8z" (OuterVolumeSpecName: "kube-api-access-4zd8z") pod "146094f8-c102-4935-9c33-3201859a59de" (UID: "146094f8-c102-4935-9c33-3201859a59de"). InnerVolumeSpecName "kube-api-access-4zd8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:20:45.718045 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.718041    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/146094f8-c102-4935-9c33-3201859a59de-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "146094f8-c102-4935-9c33-3201859a59de" (UID: "146094f8-c102-4935-9c33-3201859a59de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:20:45.718281 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.718057    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-dshm" (OuterVolumeSpecName: "dshm") pod "146094f8-c102-4935-9c33-3201859a59de" (UID: "146094f8-c102-4935-9c33-3201859a59de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:20:45.817090 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.817054    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:20:45.817090 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.817084    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/146094f8-c102-4935-9c33-3201859a59de-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:20:45.817090 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.817093    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zd8z\" (UniqueName: \"kubernetes.io/projected/146094f8-c102-4935-9c33-3201859a59de-kube-api-access-4zd8z\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:20:45.817335 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.817101    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/146094f8-c102-4935-9c33-3201859a59de-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:20:45.931205 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.931114    2572 generic.go:356] "Generic (PLEG): container finished" podID="146094f8-c102-4935-9c33-3201859a59de" containerID="2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79" exitCode=0
Jul 25 00:20:45.931205 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.931159    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"
Jul 25 00:20:45.931205 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.931180    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2" event={"ID":"146094f8-c102-4935-9c33-3201859a59de","Type":"ContainerDied","Data":"2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79"}
Jul 25 00:20:45.931469 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.931219    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2" event={"ID":"146094f8-c102-4935-9c33-3201859a59de","Type":"ContainerDied","Data":"2e343776e15e620556bc1774e3b896f8e73d50d69c72d97794a677fe9c782790"}
Jul 25 00:20:45.931469 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.931242    2572 scope.go:117] "RemoveContainer" containerID="2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79"
Jul 25 00:20:45.941226 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.941200    2572 scope.go:117] "RemoveContainer" containerID="2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79"
Jul 25 00:20:45.941550 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:20:45.941528    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79\": container with ID starting with 2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79 not found: ID does not exist" containerID="2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79"
Jul 25 00:20:45.941614 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.941559    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79"} err="failed to get container status \"2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79\": rpc error: code = NotFound desc = could not find container \"2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79\": container with ID starting with 2288cdd1f4e5138e549386ca61406b943c3bc34ffada923c0393004e36d49a79 not found: ID does not exist"
Jul 25 00:20:45.952438 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.952405    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"]
Jul 25 00:20:45.956239 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.956208    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-be7457/ipp-route-be7457-kserve-5f84c64f4c-gmxn2"]
Jul 25 00:20:45.966617 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:20:45.966574    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="146094f8-c102-4935-9c33-3201859a59de" path="/var/lib/kubelet/pods/146094f8-c102-4935-9c33-3201859a59de/volumes"
Jul 25 00:21:27.490303 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:27.490262    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:21:35.388749 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:35.388709    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"]
Jul 25 00:21:35.389198 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:35.388981    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy" containerID="cri-o://84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea" gracePeriod=30
Jul 25 00:21:40.653169 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.653087    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:21:40.764879 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.764837    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-credential-socket\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.764914    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-socket\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.764942    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-certs\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.764974    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-token\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.764992    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-podinfo\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765018    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-envoy\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765082 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765036    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istiod-ca-cert\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765425 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765187    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72fkl\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-kube-api-access-72fkl\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765425 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765197    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:21:40.765425 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765243    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:21:40.765425 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765274    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-data\") pod \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\" (UID: \"b185eeb5-f2b2-4fe8-8c66-9413c963ea52\") "
Jul 25 00:21:40.765425 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765284    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:21:40.765732 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765527    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:40.765732 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765633    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-credential-socket\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.765732 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765653    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-socket\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.765732 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.765669    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-workload-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.766183 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.766159    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-data" (OuterVolumeSpecName: "istio-data") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:21:40.767652 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.767620    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-kube-api-access-72fkl" (OuterVolumeSpecName: "kube-api-access-72fkl") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "kube-api-access-72fkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:21:40.767760 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.767659    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 00:21:40.767808 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.767755    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 00:21:40.767808 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.767765    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-token" (OuterVolumeSpecName: "istio-token") pod "b185eeb5-f2b2-4fe8-8c66-9413c963ea52" (UID: "b185eeb5-f2b2-4fe8-8c66-9413c963ea52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:21:40.866632 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.866589    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-envoy\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.866632 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.866624    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istiod-ca-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.866632 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.866634    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72fkl\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-kube-api-access-72fkl\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.866875 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.866643    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-data\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.866875 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.866653    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-token\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:40.866875 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:40.866660    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b185eeb5-f2b2-4fe8-8c66-9413c963ea52-istio-podinfo\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 25 00:21:41.212138 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.212099    2572 generic.go:356] "Generic (PLEG): container finished" podID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerID="84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea" exitCode=0
Jul 25 00:21:41.212315 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.212149    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" event={"ID":"b185eeb5-f2b2-4fe8-8c66-9413c963ea52","Type":"ContainerDied","Data":"84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea"}
Jul 25 00:21:41.212315 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.212153    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"
Jul 25 00:21:41.212315 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.212193    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp" event={"ID":"b185eeb5-f2b2-4fe8-8c66-9413c963ea52","Type":"ContainerDied","Data":"35ee34d3790f2007743bea800492fe4c3979c48e2d21a22d8a9ab65bcd5c4156"}
Jul 25 00:21:41.212315 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.212214    2572 scope.go:117] "RemoveContainer" containerID="84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea"
Jul 25 00:21:41.222103 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.222082    2572 scope.go:117] "RemoveContainer" containerID="a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2"
Jul 25 00:21:41.230516 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.230493    2572 scope.go:117] "RemoveContainer" containerID="84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea"
Jul 25 00:21:41.230801 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:21:41.230783    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea\": container with ID starting with 84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea not found: ID does not exist" containerID="84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea"
Jul 25 00:21:41.230842 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.230811    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea"} err="failed to get container status \"84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea\": rpc error: code = NotFound desc = could not find container \"84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea\": container with ID starting with 84fcb3716786cf4a8c6f5c84190a2c65d2784c038e2ded87d2f6cea0cb228eea not found: ID does not exist"
Jul 25 00:21:41.230842 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.230828    2572 scope.go:117] "RemoveContainer" containerID="a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2"
Jul 25 00:21:41.231403 ip-10-0-141-134 kubenswrapper[2572]: E0725 00:21:41.231218    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2\": container with ID starting with a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2 not found: ID does not exist" containerID="a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2"
Jul 25 00:21:41.231403 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.231289    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2"} err="failed to get container status \"a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2\": rpc error: code = NotFound desc = could not find container \"a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2\": container with ID starting with a0fe6c52f179961d620021c0138b9a0b2b6c0bd356d9171b8833c6c94f9792f2 not found: ID does not exist"
Jul 25 00:21:41.233700 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.233677    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"]
Jul 25 00:21:41.237197 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.237171    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-899559-openshift-default-865fc748b-7q9rp"]
Jul 25 00:21:41.967204 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:41.967154    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" path="/var/lib/kubelet/pods/b185eeb5-f2b2-4fe8-8c66-9413c963ea52/volumes"
Jul 25 00:21:46.053995 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:46.053956    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"]
Jul 25 00:21:46.061732 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:46.061696    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-92df89-29748975-z8xjf"]
Jul 25 00:21:47.966115 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:47.966029    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5724e25a-6f89-4dfa-8ef6-af360834ad54" path="/var/lib/kubelet/pods/5724e25a-6f89-4dfa-8ef6-af360834ad54/volumes"
Jul 25 00:21:49.187877 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:21:49.187835    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:22:12.082652 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:12.082617    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rmksn"]
Jul 25 00:22:20.514828 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:20.514785    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5688cf88c8-qn2hv_09eab24f-6c64-4613-b0e1-e13d290fd25b/authorino/0.log"
Jul 25 00:22:26.297730 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:26.297700    2572 scope.go:117] "RemoveContainer" containerID="9de81bf31829c71bc1a9e90fd69b49059e5cd1f4ae21213ebcfb772438cbbe9b"
Jul 25 00:22:27.127689 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:27.127656    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-tzlz8_823caa7f-30bd-478e-8590-92cb89fe2679/copy-manifests/0.log"
Jul 25 00:22:27.134659 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:27.134631    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-tzlz8_823caa7f-30bd-478e-8590-92cb89fe2679/manager/0.log"
Jul 25 00:22:28.032130 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:28.032101    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748975-7hfxm_8cd8e02c-d638-4848-a40d-7af70d08ed7e/cleanup/0.log"
Jul 25 00:22:29.555198 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:29.555161    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5688cf88c8-qn2hv_09eab24f-6c64-4613-b0e1-e13d290fd25b/authorino/0.log"
Jul 25 00:22:29.919819 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:29.919739    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rdt84_b511136e-36dd-43d7-8588-baf1fa5bf10a/kuadrant-console-plugin/0.log"
Jul 25 00:22:30.300676 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:30.300650    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rmksn_f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0/limitador/0.log"
Jul 25 00:22:30.835843 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:30.835806    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cz94zk_13347472-e20b-451d-9a70-2b2501dbe6d4/istio-proxy/0.log"
Jul 25 00:22:31.096944 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:31.096863    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d7d7d465-8ctbx_18e08dbc-3fb2-465d-9e6a-53f84b711c59/kube-auth-proxy/0.log"
Jul 25 00:22:31.360987 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:31.360904    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-hm8ww_0a038cf5-7cb5-47d1-88b6-c0ccef1051ec/istio-proxy/1.log"
Jul 25 00:22:32.656532 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:32.656497    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-tvtwk_a986134b-78bf-4933-a201-1a25f130a257/main/0.log"
Jul 25 00:22:32.778343 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:32.778313    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-569mk_a9d10e94-717d-4b3d-840c-eda7fdeb717d/main/0.log"
Jul 25 00:22:41.558813 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:41.558777    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n6fg5_6634d566-76cf-4c96-9668-bc6ee975ed83/global-pull-secret-syncer/0.log"
Jul 25 00:22:41.697432 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:41.697391    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nlz49_7f912844-4661-427c-aa0a-139a17222d34/konnectivity-agent/0.log"
Jul 25 00:22:41.774071 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:41.774036    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-134.ec2.internal_690e7b1c9b542cf29dc8b6b0d7b228e9/haproxy/0.log"
Jul 25 00:22:47.200678 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:47.200566    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5688cf88c8-qn2hv_09eab24f-6c64-4613-b0e1-e13d290fd25b/authorino/0.log"
Jul 25 00:22:47.272089 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:47.272061    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rdt84_b511136e-36dd-43d7-8588-baf1fa5bf10a/kuadrant-console-plugin/0.log"
Jul 25 00:22:47.527542 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:47.527510    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rmksn_f419a8ba-1e71-42bd-a51a-e5ccf9e08cf0/limitador/0.log"
Jul 25 00:22:49.551956 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.551921    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-794cb85898-g7wmw_afb7ed68-2a6a-43c3-87bb-4a6bffb48544/metrics-server/0.log"
Jul 25 00:22:49.575740 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.575706    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-br9cj_d60abb81-f81d-40d4-be40-47268704771e/monitoring-plugin/0.log"
Jul 25 00:22:49.606931 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.606898    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7q8td_ba37a67a-9c72-485c-9f82-5de223d85fcb/node-exporter/0.log"
Jul 25 00:22:49.625162 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.625130    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7q8td_ba37a67a-9c72-485c-9f82-5de223d85fcb/kube-rbac-proxy/0.log"
Jul 25 00:22:49.644343 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.644318    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7q8td_ba37a67a-9c72-485c-9f82-5de223d85fcb/init-textfile/0.log"
Jul 25 00:22:49.821495 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.821394    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-jbfbn_c229b15b-5001-412f-9027-62e76f2396e1/kube-rbac-proxy-main/0.log"
Jul 25 00:22:49.840014 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.839984    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-jbfbn_c229b15b-5001-412f-9027-62e76f2396e1/kube-rbac-proxy-self/0.log"
Jul 25 00:22:49.860989 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.860944    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-jbfbn_c229b15b-5001-412f-9027-62e76f2396e1/openshift-state-metrics/0.log"
Jul 25 00:22:49.896969 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.896934    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62ad74ea-fb39-47b7-87f9-e8f9374c33aa/prometheus/0.log"
Jul 25 00:22:49.914854 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.914820    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62ad74ea-fb39-47b7-87f9-e8f9374c33aa/config-reloader/0.log"
Jul 25 00:22:49.934875 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:49.934850    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62ad74ea-fb39-47b7-87f9-e8f9374c33aa/thanos-sidecar/0.log"
Jul 25 00:22:50.009382 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.009357    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62ad74ea-fb39-47b7-87f9-e8f9374c33aa/kube-rbac-proxy-web/0.log"
Jul 25 00:22:50.064267 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.064237    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62ad74ea-fb39-47b7-87f9-e8f9374c33aa/kube-rbac-proxy/0.log"
Jul 25 00:22:50.089152 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.089058    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62ad74ea-fb39-47b7-87f9-e8f9374c33aa/kube-rbac-proxy-thanos/0.log"
Jul 25 00:22:50.115650 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.115621    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_62ad74ea-fb39-47b7-87f9-e8f9374c33aa/init-config-reloader/0.log"
Jul 25 00:22:50.202386 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.202355    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7665fd9b8d-zcrhj_738bdb77-2ca6-49de-9854-d3b8d6212866/telemeter-client/0.log"
Jul 25 00:22:50.221612 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.221581    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7665fd9b8d-zcrhj_738bdb77-2ca6-49de-9854-d3b8d6212866/reload/0.log"
Jul 25 00:22:50.240817 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.240787    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7665fd9b8d-zcrhj_738bdb77-2ca6-49de-9854-d3b8d6212866/kube-rbac-proxy/0.log"
Jul 25 00:22:50.268916 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.268882    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dfbbf9f65-s75k4_dba4e419-9c48-496d-a88b-154683c1338a/thanos-query/0.log"
Jul 25 00:22:50.288609 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.288578    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dfbbf9f65-s75k4_dba4e419-9c48-496d-a88b-154683c1338a/kube-rbac-proxy-web/0.log"
Jul 25 00:22:50.307101 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.307071    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dfbbf9f65-s75k4_dba4e419-9c48-496d-a88b-154683c1338a/kube-rbac-proxy/0.log"
Jul 25 00:22:50.325790 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.325760    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dfbbf9f65-s75k4_dba4e419-9c48-496d-a88b-154683c1338a/prom-label-proxy/0.log"
Jul 25 00:22:50.344599 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.344516    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dfbbf9f65-s75k4_dba4e419-9c48-496d-a88b-154683c1338a/kube-rbac-proxy-rules/0.log"
Jul 25 00:22:50.363894 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:50.363864    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dfbbf9f65-s75k4_dba4e419-9c48-496d-a88b-154683c1338a/kube-rbac-proxy-metrics/0.log"
Jul 25 00:22:51.503013 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.502970    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"]
Jul 25 00:22:51.503429 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503415    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e06bac-c805-4e6e-ba2e-939a85ae05a3" containerName="revoke-keys"
Jul 25 00:22:51.503429 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503431    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e06bac-c805-4e6e-ba2e-939a85ae05a3" containerName="revoke-keys"
Jul 25 00:22:51.503543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503441    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy"
Jul 25 00:22:51.503543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503447    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy"
Jul 25 00:22:51.503543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503472    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy"
Jul 25 00:22:51.503543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503479    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy"
Jul 25 00:22:51.503543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503497    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146094f8-c102-4935-9c33-3201859a59de" containerName="main"
Jul 25 00:22:51.503543 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503503    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="146094f8-c102-4935-9c33-3201859a59de" containerName="main"
Jul 25 00:22:51.503729 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503569    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b185eeb5-f2b2-4fe8-8c66-9413c963ea52" containerName="istio-proxy"
Jul 25 00:22:51.503729 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503575    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="146094f8-c102-4935-9c33-3201859a59de" containerName="main"
Jul 25 00:22:51.503729 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.503579    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e06bac-c805-4e6e-ba2e-939a85ae05a3" containerName="revoke-keys"
Jul 25 00:22:51.507019 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.506994    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.509528 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.509495    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t745v\"/\"default-dockercfg-f6z68\""
Jul 25 00:22:51.510393 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.510369    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t745v\"/\"kube-root-ca.crt\""
Jul 25 00:22:51.510603 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.510382    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t745v\"/\"openshift-service-ca.crt\""
Jul 25 00:22:51.515706 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.515681    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"]
Jul 25 00:22:51.621041 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.620995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbkq9\" (UniqueName: \"kubernetes.io/projected/43a7c249-63ad-44f7-a992-b0bfe4c72e76-kube-api-access-sbkq9\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.621514 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.621136    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-proc\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.621514 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.621189    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-sys\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.621514 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.621219    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-podres\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.621514 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.621235    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-lib-modules\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722544 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722494    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-proc\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722544 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722552    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-sys\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722583    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-podres\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722606    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-lib-modules\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722636    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-proc\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722663    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbkq9\" (UniqueName: \"kubernetes.io/projected/43a7c249-63ad-44f7-a992-b0bfe4c72e76-kube-api-access-sbkq9\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722700    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-sys\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722763    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-podres\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.722826 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.722773    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43a7c249-63ad-44f7-a992-b0bfe4c72e76-lib-modules\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.730316 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.730287    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbkq9\" (UniqueName: \"kubernetes.io/projected/43a7c249-63ad-44f7-a992-b0bfe4c72e76-kube-api-access-sbkq9\") pod \"perf-node-gather-daemonset-g2kcf\" (UID: \"43a7c249-63ad-44f7-a992-b0bfe4c72e76\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.818871 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.818762    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:51.960266 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:51.960236    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"]
Jul 25 00:22:51.962682 ip-10-0-141-134 kubenswrapper[2572]: W0725 00:22:51.962651    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod43a7c249_63ad_44f7_a992_b0bfe4c72e76.slice/crio-7202a21ba89fc2f9b4b57c6e231d491fff67f28734123341f7acc61c8224f4cb WatchSource:0}: Error finding container 7202a21ba89fc2f9b4b57c6e231d491fff67f28734123341f7acc61c8224f4cb: Status 404 returned error can't find the container with id 7202a21ba89fc2f9b4b57c6e231d491fff67f28734123341f7acc61c8224f4cb
Jul 25 00:22:52.571869 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:52.571833    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf" event={"ID":"43a7c249-63ad-44f7-a992-b0bfe4c72e76","Type":"ContainerStarted","Data":"104a443954d241ebb96208310f67b40b8bd095c6688353b76b096b3bf31caaab"}
Jul 25 00:22:52.572290 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:52.571886    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf" event={"ID":"43a7c249-63ad-44f7-a992-b0bfe4c72e76","Type":"ContainerStarted","Data":"7202a21ba89fc2f9b4b57c6e231d491fff67f28734123341f7acc61c8224f4cb"}
Jul 25 00:22:52.572290 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:52.571974    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:22:52.586832 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:52.586772    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf" podStartSLOduration=1.5867538030000001 podStartE2EDuration="1.586753803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:22:52.585970388 +0000 UTC m=+3207.081682264" watchObservedRunningTime="2026-07-25 00:22:52.586753803 +0000 UTC m=+3207.082465691"
Jul 25 00:22:52.623550 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:52.623513    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-b767d67fb-8zclz_5bd1eda3-6d18-4211-81da-ab5e71505234/console/0.log"
Jul 25 00:22:54.219021 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:54.218983    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dhmkj_e929bc11-c049-490a-a60e-48be763e1b27/dns/0.log"
Jul 25 00:22:54.238338 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:54.238306    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dhmkj_e929bc11-c049-490a-a60e-48be763e1b27/kube-rbac-proxy/0.log"
Jul 25 00:22:54.388491 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:54.388434    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c427z_608f5875-7843-48a2-a100-6050a0fdac6c/dns-node-resolver/0.log"
Jul 25 00:22:54.958737 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:54.958706    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6fwbl_b76e78ba-3363-4b80-9ac2-0ff25f65088f/node-ca/0.log"
Jul 25 00:22:55.921037 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:55.921007    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cz94zk_13347472-e20b-451d-9a70-2b2501dbe6d4/istio-proxy/0.log"
Jul 25 00:22:56.201302 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:56.201215    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d7d7d465-8ctbx_18e08dbc-3fb2-465d-9e6a-53f84b711c59/kube-auth-proxy/0.log"
Jul 25 00:22:56.254776 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:56.254746    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-hm8ww_0a038cf5-7cb5-47d1-88b6-c0ccef1051ec/istio-proxy/0.log"
Jul 25 00:22:56.312957 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:56.312917    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-hm8ww_0a038cf5-7cb5-47d1-88b6-c0ccef1051ec/istio-proxy/1.log"
Jul 25 00:22:57.184411 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:57.184377    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-84pqs_648fb8e3-226a-422a-8e0b-cbdf0e58a310/serve-healthcheck-canary/0.log"
Jul 25 00:22:57.766179 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:57.766145    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fg5dg_46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f/kube-rbac-proxy/0.log"
Jul 25 00:22:57.784100 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:57.784066    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fg5dg_46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f/exporter/0.log"
Jul 25 00:22:57.802781 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:57.802751    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fg5dg_46b4d986-fc79-4c4d-ba2c-2b6e4f56ac0f/extractor/0.log"
Jul 25 00:22:58.587231 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:22:58.587203    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-g2kcf"
Jul 25 00:23:00.894388 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:00.894343    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-tzlz8_823caa7f-30bd-478e-8590-92cb89fe2679/manager/0.log"
Jul 25 00:23:00.915660 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:00.915633    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-tzlz8_823caa7f-30bd-478e-8590-92cb89fe2679/copy-manifests/0.log"
Jul 25 00:23:02.187466 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:02.187421    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cz94zk_13347472-e20b-451d-9a70-2b2501dbe6d4/istio-proxy/0.log"
Jul 25 00:23:02.285136 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:02.285108    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54fc5fd468-pkvjl_9764f72c-6097-42a3-8e2b-cddd717eca45/manager/0.log"
Jul 25 00:23:02.458708 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:02.458616    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d7d7d465-8ctbx_18e08dbc-3fb2-465d-9e6a-53f84b711c59/kube-auth-proxy/0.log"
Jul 25 00:23:02.492498 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:02.492443    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-hm8ww_0a038cf5-7cb5-47d1-88b6-c0ccef1051ec/istio-proxy/0.log"
Jul 25 00:23:02.544683 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:02.544648    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-hm8ww_0a038cf5-7cb5-47d1-88b6-c0ccef1051ec/istio-proxy/1.log"
Jul 25 00:23:03.948254 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:03.948223    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cz94zk_13347472-e20b-451d-9a70-2b2501dbe6d4/istio-proxy/0.log"
Jul 25 00:23:04.221989 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:04.221896    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d7d7d465-8ctbx_18e08dbc-3fb2-465d-9e6a-53f84b711c59/kube-auth-proxy/0.log"
Jul 25 00:23:04.256593 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:04.256555    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-hm8ww_0a038cf5-7cb5-47d1-88b6-c0ccef1051ec/istio-proxy/0.log"
Jul 25 00:23:04.311231 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:04.311193    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-hm8ww_0a038cf5-7cb5-47d1-88b6-c0ccef1051ec/istio-proxy/1.log"
Jul 25 00:23:08.524060 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:08.524027    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdcv2_c4cac730-5c82-4f69-a783-34f405316b30/kube-multus-additional-cni-plugins/0.log"
Jul 25 00:23:08.542570 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:08.542543    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdcv2_c4cac730-5c82-4f69-a783-34f405316b30/egress-router-binary-copy/0.log"
Jul 25 00:23:08.560341 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:08.560312    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdcv2_c4cac730-5c82-4f69-a783-34f405316b30/cni-plugins/0.log"
Jul 25 00:23:08.577468 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:08.577420    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdcv2_c4cac730-5c82-4f69-a783-34f405316b30/bond-cni-plugin/0.log"
Jul 25 00:23:08.595197 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:08.595147    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdcv2_c4cac730-5c82-4f69-a783-34f405316b30/routeoverride-cni/0.log"
Jul 25 00:23:08.613197 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:08.613163    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdcv2_c4cac730-5c82-4f69-a783-34f405316b30/whereabouts-cni-bincopy/0.log"
Jul 25 00:23:08.631259 ip-10-0-141-134 kubenswrapper[2572]: I0725 00:23:08.631227    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdcv2_c4cac730-5c82-4f69-a783-34f405316b30/whereabouts-cni/0.log"