Jul 15 15:24:33.903517 ip-10-0-135-125 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.320267    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325812    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325825    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325829    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325832    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325835    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325838    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325842    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 15:24:34.352150 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325845    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325848    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325851    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325853    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325856    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325859    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325861    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325864    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325866    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325869    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325872    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325876    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325878    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325888    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325891    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325893    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325896    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325899    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325901    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325904    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 15:24:34.352983 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325906    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325909    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325912    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325915    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325918    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325920    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325923    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325925    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325927    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325930    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325933    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325935    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325938    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325941    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325943    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325946    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325949    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325951    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325954    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325956    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 15:24:34.353513 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325958    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325963    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325965    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325968    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325970    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325973    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325975    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325978    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325981    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325983    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325986    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325988    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325991    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325993    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325996    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.325998    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326001    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326003    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326006    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326008    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 15:24:34.354123 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326011    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326014    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326019    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326023    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326026    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326029    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326032    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326034    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326037    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326040    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326042    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326045    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326048    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326050    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326053    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326055    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326057    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326060    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326063    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 15:24:34.354698 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326065    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326068    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326070    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326074    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326077    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326080    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326083    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326086    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326089    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326092    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326094    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326097    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326100    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326838    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326845    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326849    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326852    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326855    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326858    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326861    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 15:24:34.355244 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326863    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326866    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326870    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326874    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326876    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326879    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326882    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326886    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326890    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326893    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326896    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326899    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326901    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326904    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326906    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326909    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326911    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326914    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326916    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 15:24:34.355772 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326931    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326935    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326939    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326942    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326945    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326948    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326951    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326954    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326957    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326960    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326963    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326965    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326969    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326971    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326974    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326976    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326978    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326981    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326983    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 15:24:34.356362 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326986    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326990    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326993    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326996    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.326998    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327001    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327003    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327006    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327008    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327011    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327014    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327017    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327021    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327024    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327027    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327030    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327033    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327035    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327038    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327040    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 15:24:34.356901 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327043    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327045    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327047    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327050    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327053    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327055    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327058    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327061    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327063    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327066    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327068    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327071    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327074    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327076    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327079    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327081    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327084    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327086    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327089    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 15:24:34.357496 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327091    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327094    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327096    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327101    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327104    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327107    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327110    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327113    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327115    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327118    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327120    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327123    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327125    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327128    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327130    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327562    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327569    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327573    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327575    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 15:24:34.357994 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327579    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327581    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327584    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327587    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327590    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327593    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327596    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327598    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327601    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327604    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327606    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327608    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327611    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327613    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327616    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327619    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327621    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327624    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327626    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327629    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 15:24:34.358584 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327631    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327634    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327637    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327639    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327642    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327644    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327647    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327649    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327652    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327654    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327658    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327661    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327665    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327668    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327670    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327673    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327676    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327681    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327683    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327686    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 15:24:34.359182 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327689    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327692    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327695    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327698    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327700    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327703    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327706    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327709    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327711    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327714    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327717    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327720    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327722    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327726    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327729    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327733    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327736    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327739    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327742    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327745    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 15:24:34.359852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327748    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327750    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327753    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327757    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327760    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327763    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327766    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327769    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327771    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327775    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327778    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327780    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327783    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327785    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327788    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327790    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327793    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327796    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327798    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327801    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 15:24:34.360521 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327804    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327806    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327809    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327811    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327814    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327817    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327822    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327825    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327827    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327830    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327833    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327836    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327839    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327841    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.327844    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327907    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327920    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327926    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327930    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327935    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 15:24:35.694873 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327938    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 15:24:34.401096 ip-10-0-135-125 systemd[1]: Started Kubernetes Kubelet.
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327944    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327948    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327951    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327954    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327957    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327960    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327963    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327966    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327969    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327972    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327975    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327978    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327981    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327984    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327987    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327990    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327993    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.327997    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328000    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328003    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328007    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328010    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328012    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 15:24:35.715356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328015    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328018    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328032    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328037    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328040    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328043    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328046    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328049    2580 flags.go:64] FLAG: --enable-server="true"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328052    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328058    2580 flags.go:64] FLAG: --event-burst="100"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328061    2580 flags.go:64] FLAG: --event-qps="50"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328064    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328067    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328070    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328073    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328077    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328080    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328083    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328086    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328088    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328091    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328094    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328097    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328100    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328103    2580 flags.go:64] FLAG: --feature-gates=""
Jul 15 15:24:35.716267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328107    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328110    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328112    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328115    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328118    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328121    2580 flags.go:64] FLAG: --help="false"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328124    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-135-125.ec2.internal"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328127    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328130    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328133    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328138    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328141    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328145    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328147    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328150    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328153    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328157    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328161    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328164    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328166    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328169    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328172    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328175    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328178    2580 flags.go:64] FLAG: --lock-file=""
Jul 15 15:24:35.717116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328181    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328183    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328186    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328192    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328194    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328197    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328200    2580 flags.go:64] FLAG: --logging-format="text"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328203    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328206    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328209    2580 flags.go:64] FLAG: --manifest-url=""
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328211    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328229    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328232    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328236    2580 flags.go:64] FLAG: --max-pods="110"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328239    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328242    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328245    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328247    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328252    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328254    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328257    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328264    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328267    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328270    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 15:24:35.717996 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328274    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328278    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328283    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328286    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328289    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328292    2580 flags.go:64] FLAG: --port="10250"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328295    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328298    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-092cc5a9e6ff537e4"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328301    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328304    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328307    2580 flags.go:64] FLAG: --register-node="true"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328310    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328313    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328316    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328319    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328321    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328325    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328328    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328331    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328333    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328336    2580 flags.go:64] FLAG: --runonce="false"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328339    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328342    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328345    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328348    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 15:24:35.719049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328350    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328354    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328358    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328361    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328363    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328367    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328370    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328372    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328378    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328381    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328384    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328389    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328392    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328395    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328399    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328401    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328404    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328407    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328410    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328413    2580 flags.go:64] FLAG: --v="2"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328416    2580 flags.go:64] FLAG: --version="false"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328423    2580 flags.go:64] FLAG: --vmodule=""
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328426    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328430    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328550    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 15:24:35.719830 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328554    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328557    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328560    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328563    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328566    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328568    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328571    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328574    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328578    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328581    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328583    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328586    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328588    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328591    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328593    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328597    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328600    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328603    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328606    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328608    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 15:24:35.970899 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328611    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328613    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328616    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328618    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328621    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328623    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328625    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328628    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328631    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328633    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328636    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328638    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328641    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328643    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328646    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328650    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328654    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328658    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328661    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 15:24:35.971564 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328664    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328668    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328671    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328673    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328676    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328679    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328681    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328683    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328687    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328691    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328693    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328696    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328699    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328701    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328704    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328707    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328709    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328711    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328714    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328716    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 15:24:35.972216 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328719    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328722    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328724    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328727    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328729    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328732    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328734    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328737    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328740    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328742    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328745    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328748    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328750    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328754    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328757    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328759    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328762    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328764    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328767    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328769    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 15:24:35.972862 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328773    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328775    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328778    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328780    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328783    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328786    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328788    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328792    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328796    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328798    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328801    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328803    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328806    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328808    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328811    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328813    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328816    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328818    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 15:24:35.973445 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.328821    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.328826    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.337869    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.337887    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.337906    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.337916    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337971    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337977    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337980    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337983    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337988    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337992    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337996    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.337999    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 15:24:35.974026 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338002    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338005    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338008    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338011    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338014    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338017    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338020    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338022    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338025    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338028    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338030    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338033    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338038    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338041    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338044    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338047    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338049    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338052    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338054    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 15:24:35.974470 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338057    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338060    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338065    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338068    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338070    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338073    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338075    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338078    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338080    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338083    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338086    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338088    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338091    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338093    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338096    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338098    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338101    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338106    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338109    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338111    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 15:24:35.975077 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338114    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338116    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338119    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338122    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338125    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338127    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338130    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338132    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338134    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338137    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338140    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338142    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338145    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338147    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338151    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338153    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338156    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338159    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338161    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338164    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 15:24:35.975646 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338166    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338169    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338172    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338175    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338177    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338180    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338184    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338188    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338191    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338195    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338198    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338201    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338204    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338207    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338210    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338213    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338230    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338233    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338236    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 15:24:35.976262 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338239    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338242    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338244    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338247    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338250    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338252    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338255    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338259    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338262    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338264    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338267    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338270    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338272    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.338277    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338388    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 15:24:35.976801 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338393    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338396    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338399    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338402    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338404    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338407    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338410    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338414    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338417    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338420    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338423    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338426    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338428    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338431    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338434    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338436    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338440    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338443    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338446    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 15:24:35.977250 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338449    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338452    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338455    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338458    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338461    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338463    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338468    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338471    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338474    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338476    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338479    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338482    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338484    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338487    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338490    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338492    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338495    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338497    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338500    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338502    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338506    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 15:24:35.977742 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338509    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338511    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338514    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338516    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338520    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338522    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338525    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338527    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338530    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338532    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338535    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338538    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338540    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338543    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338545    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338547    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338550    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338554    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338556    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 15:24:35.978351 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338559    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338561    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338564    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338566    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338569    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338571    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338574    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338576    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338579    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338581    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338584    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338586    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338590    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338593    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338595    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338598    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338601    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338603    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338607    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338609    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 15:24:35.978852 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338612    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338614    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338617    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338619    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338622    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338626    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338630    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338634    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338638    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338641    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338645    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338648    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338651    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338653    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338656    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338659    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338661    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338664    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 15:24:35.979404 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:34.338667    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.338672    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.338780    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.340727    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.342184    2580 server.go:1019] "Starting client certificate rotation"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.342250    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.342281    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.344570    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.352753    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.364586    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.368717    2580 log.go:25] "Validated CRI v1 image API"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.369584    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 15:24:35.979941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.372261    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 15:24:35.980319 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.373933    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b4f614b2-46af-421b-8b52-096e4cb2251a:/dev/nvme0n1p4 b77594ce-8ea0-4680-9364-525e1b3ba8ca:/dev/nvme0n1p3]
Jul 15 15:24:35.980319 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.373950    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.382154    2580 manager.go:216] Machine: {Timestamp:2026-07-15 15:24:34.378384769 +0000 UTC m=+0.348377186 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099446 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ea01768383904d7db8e13f595ec87 SystemUUID:ec2ea017-6838-3904-d7db-8e13f595ec87 BootID:c521218b-fa9f-414b-af80-5d8f7997dc1e Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a3:cc:21:4d:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a3:cc:21:4d:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:68:a2:03:a2:96 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.382264    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.382339    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.385071    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.385097    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-125.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.385870    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.385881    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.385889    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.386632    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.387347    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390429    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390534    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390559    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390567    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390580    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390589    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390597    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 15:24:35.980487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390888    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.390902    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.392841    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394252    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394282    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394880    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394893    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394899    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394905    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394913    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394918    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.394996    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.395002    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.395009    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.395014    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.395023    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.395030    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.396894    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.396904    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.400249    2580 server.go:1268] "Started kubelet"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.400349    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.400348    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.400423    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.401454    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-125.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.401484    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-125.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.401520    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.401640    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.403166    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.406349    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.405836    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-125.ec2.internal.18c27ff0c1121b0e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-125.ec2.internal,UID:ip-10-0-135-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-125.ec2.internal,},FirstTimestamp:2026-07-15 15:24:34.400205582 +0000 UTC m=+0.370197998,LastTimestamp:2026-07-15 15:24:34.400205582 +0000 UTC m=+0.370197998,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-125.ec2.internal,}"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.407147    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.407165    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.407356    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.407908    2580 factory.go:55] Registering systemd factory
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.407921    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408014    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408011    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408058    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408121    2580 factory.go:153] Registering CRI-O factory
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408133    2580 factory.go:223] Registration of the crio container factory successfully
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408138    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 15:24:35.981308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408144    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408182    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408231    2580 factory.go:103] Registering Raw factory
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.408247    2580 manager.go:1196] Started watching for new ooms in manager
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.409160    2580 manager.go:349] Starting recovery of all containers
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.409826    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4c7cq"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.417029    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4c7cq"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.418172    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-125.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.418270    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.418737    2580 manager.go:354] Recovery completed
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.423951    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.425575    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.425598    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.425607    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.426982    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.427131    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.427149    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.426995    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.432890    2580 policy_none.go:49] "None policy: Start"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.433417    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.433443    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.433460    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.433906    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.433920    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.435533    2580 policy_none.go:47] "Start"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.479774    2580 manager.go:337] "Starting Device Plugin manager"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.479800    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.479809    2580 server.go:87] "Starting device plugin registration server"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.480028    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.480038    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.480162    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.480263    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:34.480273    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.480774    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:34.480812    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.749176    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.750156    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.750182    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.750197    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.750226    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-125.ec2.internal"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.759721    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.760820    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-125.ec2.internal"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:35.760839    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-125.ec2.internal\": node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.760888    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.760916    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 15:24:35.982440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.760933    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:35.760969    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.765667    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:35.801280    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.861976    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"]
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.862092    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.863921    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.863947    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.863957    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.865644    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.865771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.865811    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.866316    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.866337    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.866346    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.866369    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.866385    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.866394    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.867666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.867687    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.868332    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.868355    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.868365    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.877693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b07821de4720a9cf3f16450e3c5031e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-125.ec2.internal\" (UID: \"7b07821de4720a9cf3f16450e3c5031e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.877714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c335c55369566ce69dad6729899f194a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"c335c55369566ce69dad6729899f194a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.877732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c335c55369566ce69dad6729899f194a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"c335c55369566ce69dad6729899f194a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:35.889348    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-125.ec2.internal\" not found" node="ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:35.893270    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-125.ec2.internal\" not found" node="ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:35.901490    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.977993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c335c55369566ce69dad6729899f194a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"c335c55369566ce69dad6729899f194a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.251295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.978039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c335c55369566ce69dad6729899f194a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"c335c55369566ce69dad6729899f194a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.978076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b07821de4720a9cf3f16450e3c5031e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-125.ec2.internal\" (UID: \"7b07821de4720a9cf3f16450e3c5031e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.978088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c335c55369566ce69dad6729899f194a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"c335c55369566ce69dad6729899f194a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.978093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c335c55369566ce69dad6729899f194a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"c335c55369566ce69dad6729899f194a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:35.978118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b07821de4720a9cf3f16450e3c5031e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-125.ec2.internal\" (UID: \"7b07821de4720a9cf3f16450e3c5031e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:36.002448    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:36.102997    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.192547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.196746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.252853 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:36.203846    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.503373 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:36.304401    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.503373 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:36.404985    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:36.505552    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.509687    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.509830    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.562176    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.576124    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.576149    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.589600    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.591373    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.593209    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.596100    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 15:19:35 +0000 UTC" deadline="2028-03-04 02:57:39.848849371 +0000 UTC"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.596128    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14339h33m3.252724263s"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.603270    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.611102    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pqtx5"
Jul 15 15:24:36.731511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.620944    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pqtx5"
Jul 15 15:24:36.734170 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:36.733974    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b07821de4720a9cf3f16450e3c5031e.slice/crio-32e71499db6b538f2395a3e87f70dfd40807221237399607a9b2de4b45a77c59 WatchSource:0}: Error finding container 32e71499db6b538f2395a3e87f70dfd40807221237399607a9b2de4b45a77c59: Status 404 returned error can't find the container with id 32e71499db6b538f2395a3e87f70dfd40807221237399607a9b2de4b45a77c59
Jul 15 15:24:36.734555 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:36.734278    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc335c55369566ce69dad6729899f194a.slice/crio-d915f070f0c47a3277cf62c4291c337be4769da6a81a67b8e7a00e2e4696f92d WatchSource:0}: Error finding container d915f070f0c47a3277cf62c4291c337be4769da6a81a67b8e7a00e2e4696f92d: Status 404 returned error can't find the container with id d915f070f0c47a3277cf62c4291c337be4769da6a81a67b8e7a00e2e4696f92d
Jul 15 15:24:36.957716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.738775    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 15:24:36.957716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.765303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" event={"ID":"c335c55369566ce69dad6729899f194a","Type":"ContainerStarted","Data":"d915f070f0c47a3277cf62c4291c337be4769da6a81a67b8e7a00e2e4696f92d"}
Jul 15 15:24:36.957716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.766134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal" event={"ID":"7b07821de4720a9cf3f16450e3c5031e","Type":"ContainerStarted","Data":"32e71499db6b538f2395a3e87f70dfd40807221237399607a9b2de4b45a77c59"}
Jul 15 15:24:36.957716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.801591    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 15:24:36.957716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:36.876949    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.560330    2580 apiserver.go:52] "Watching apiserver"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.569919    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.570535    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vlhsk","kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn","openshift-multus/multus-additional-cni-plugins-47z5k","openshift-multus/network-metrics-daemon-zkn9v","openshift-network-diagnostics/network-check-target-l87mx","openshift-network-operator/iptables-alerter-6cwzz","openshift-ovn-kubernetes/ovnkube-node-m46hw","kube-system/konnectivity-agent-mwv45","openshift-cluster-node-tuning-operator/tuned-b8rmp","openshift-dns/node-resolver-x5qct","openshift-image-registry/node-ca-g2qfq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal","openshift-multus/multus-f9gnk"]
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.572215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.573666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.575007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.575274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.575606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.575662    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.578279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.580263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6d5nf\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.580318    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.580415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.580430    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.580443    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.580559    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.580683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.580771    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.581600    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.581718    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.581838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qwf6k\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.582074    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.582134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bfqhk\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.582252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-84qt5\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.582075    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.582344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.582387    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.582444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.582499    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.584144    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.619161 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.584830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-run-netns\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-env-overrides\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-cnibin\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-etc-kubernetes\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-var-lib-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-cni-netd\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-netns\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30334585-bcd0-44b9-bdf0-0677a905c124-konnectivity-ca\") pod \"konnectivity-agent-mwv45\" (UID: \"30334585-bcd0-44b9-bdf0-0677a905c124\") " pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-ovn\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-etc-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysconfig\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30334585-bcd0-44b9-bdf0-0677a905c124-agent-certs\") pod \"konnectivity-agent-mwv45\" (UID: \"30334585-bcd0-44b9-bdf0-0677a905c124\") " pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-kubelet\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-k8s-cni-cncf-io\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.621406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-conf-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-cni-bin\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-sys\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w2rb\" (UniqueName: \"kubernetes.io/projected/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-kube-api-access-2w2rb\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk82f\" (UniqueName: \"kubernetes.io/projected/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-kube-api-access-fk82f\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-systemd\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-run\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-lib-modules\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.585992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-cni-binary-copy\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-node-log\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmn9q\" (UniqueName: \"kubernetes.io/projected/9a15276c-947b-43db-abc2-d95c4025e505-kube-api-access-xmn9q\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-os-release\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-cni-bin\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysctl-conf\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-socket-dir-parent\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-slash\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-system-cni-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.622421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-modprobe-d\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-host\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-systemd\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-hostroot\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.586561    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-log-socket\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-ovnkube-config\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-cni-multus\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkmkq\" (UniqueName: \"kubernetes.io/projected/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-kube-api-access-zkmkq\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.586973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a15276c-947b-43db-abc2-d95c4025e505-ovn-node-metrics-cert\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-kubelet\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-multus-certs\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587235    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-ovnkube-script-lib\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-kubernetes\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-var-lib-kubelet\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.623361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-tmp\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587452    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kspfz\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587545    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-systemd-units\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-run-ovn-kubernetes\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-cni-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-daemon-config\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysctl-d\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.587863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-tuned\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.588032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cjt29\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.588218    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.588305    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.588326    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.589900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.595372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.595633    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.595723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-88s66\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.595865    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.595950    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.596686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.610137    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.610166    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.610356    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.610410    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.610471    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kpjbn\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.610620    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mdpqs\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.611154    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.622606    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.622637    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 15:19:36 +0000 UTC" deadline="2028-05-01 20:54:40.939555274 +0000 UTC"
Jul 15 15:24:37.624398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.622691    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15749h30m3.316867526s"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.677766    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-ovn\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-etc-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysconfig\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-etc-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-ovn\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30334585-bcd0-44b9-bdf0-0677a905c124-agent-certs\") pod \"konnectivity-agent-mwv45\" (UID: \"30334585-bcd0-44b9-bdf0-0677a905c124\") " pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysconfig\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/baefe377-2257-4a6a-855b-231cd6cd313a-iptables-alerter-script\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-cnibin\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-cni-binary-copy\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-sys-fs\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-kubelet\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.688990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.733041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-k8s-cni-cncf-io\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-conf-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-socket-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-cni-bin\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-sys\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w2rb\" (UniqueName: \"kubernetes.io/projected/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-kube-api-access-2w2rb\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ca195cd-f793-4606-b4d9-a8048bc5e102-dbus\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hz4r\" (UniqueName: \"kubernetes.io/projected/baeece6e-85ae-48ff-8df3-50336c6cc138-kube-api-access-2hz4r\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-k8s-cni-cncf-io\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk82f\" (UniqueName: \"kubernetes.io/projected/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-kube-api-access-fk82f\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-cni-bin\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-systemd\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-conf-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-run\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689217    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-lib-modules\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-lib-modules\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-run\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.734970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-sys\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-systemd\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-cni-binary-copy\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-node-log\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmn9q\" (UniqueName: \"kubernetes.io/projected/9a15276c-947b-43db-abc2-d95c4025e505-kube-api-access-xmn9q\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-kubelet\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-os-release\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-node-log\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-cni-bin\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.689992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-os-release\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysctl-conf\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-cni-bin\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-socket-dir-parent\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-os-release\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-device-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysctl-conf\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.736064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-socket-dir-parent\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-slash\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-system-cni-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-modprobe-d\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-host\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-slash\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-system-cni-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-hosts-file\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8mkx\" (UniqueName: \"kubernetes.io/projected/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-kube-api-access-j8mkx\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-cni-binary-copy\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-host\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-systemd\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-hostroot\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-log-socket\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-modprobe-d\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-hostroot\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.737179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-systemd\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.690467    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-log-socket\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-ovnkube-config\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.690554    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs podName:3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:38.190515824 +0000 UTC m=+2.991975616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs") pod "network-metrics-daemon-zkn9v" (UID: "3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-cni-multus\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-cni-multus\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkmkq\" (UniqueName: \"kubernetes.io/projected/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-kube-api-access-zkmkq\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r7mw\" (UniqueName: \"kubernetes.io/projected/baefe377-2257-4a6a-855b-231cd6cd313a-kube-api-access-6r7mw\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ca195cd-f793-4606-b4d9-a8048bc5e102-kubelet-config\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-serviceca\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlmp9\" (UniqueName: \"kubernetes.io/projected/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-kube-api-access-vlmp9\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a15276c-947b-43db-abc2-d95c4025e505-ovn-node-metrics-cert\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-kubelet\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-multus-certs\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-registration-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.738091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-multus-certs\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-var-lib-kubelet\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.690970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-ovnkube-script-lib\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-ovnkube-config\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-kubernetes\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-tuning-conf-dir\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-kubernetes\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw8bc\" (UniqueName: \"kubernetes.io/projected/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-kube-api-access-bw8bc\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-var-lib-kubelet\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-tmp\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-systemd-units\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-var-lib-kubelet\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-run-ovn-kubernetes\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-cni-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-systemd-units\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-daemon-config\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysctl-d\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-tuned\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-tmp-dir\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-system-cni-dir\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-sysctl-d\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-run-netns\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-env-overrides\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-run-ovn-kubernetes\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-ovnkube-script-lib\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-cnibin\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-etc-kubernetes\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-var-lib-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-cni-dir\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-cni-netd\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-netns\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-multus-daemon-config\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.739865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30334585-bcd0-44b9-bdf0-0677a905c124-konnectivity-ca\") pod \"konnectivity-agent-mwv45\" (UID: \"30334585-bcd0-44b9-bdf0-0677a905c124\") " pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-etc-kubernetes\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/baefe377-2257-4a6a-855b-231cd6cd313a-host-slash\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-host\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-run-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-run-netns\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-var-lib-openvswitch\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a15276c-947b-43db-abc2-d95c4025e505-host-cni-netd\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.691993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-host-run-netns\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.692032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-cnibin\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.692063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a15276c-947b-43db-abc2-d95c4025e505-env-overrides\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.692414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/30334585-bcd0-44b9-bdf0-0677a905c124-konnectivity-ca\") pod \"konnectivity-agent-mwv45\" (UID: \"30334585-bcd0-44b9-bdf0-0677a905c124\") " pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.702043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/30334585-bcd0-44b9-bdf0-0677a905c124-agent-certs\") pod \"konnectivity-agent-mwv45\" (UID: \"30334585-bcd0-44b9-bdf0-0677a905c124\") " pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.702586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-tmp\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.703634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a15276c-947b-43db-abc2-d95c4025e505-ovn-node-metrics-cert\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.704325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w2rb\" (UniqueName: \"kubernetes.io/projected/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-kube-api-access-2w2rb\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.704715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkmkq\" (UniqueName: \"kubernetes.io/projected/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-kube-api-access-zkmkq\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.705067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb-etc-tuned\") pod \"tuned-b8rmp\" (UID: \"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb\") " pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:37.742631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.705095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk82f\" (UniqueName: \"kubernetes.io/projected/1a3149f3-f5d7-40f0-921f-6f78f88b88ff-kube-api-access-fk82f\") pod \"multus-f9gnk\" (UID: \"1a3149f3-f5d7-40f0-921f-6f78f88b88ff\") " pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:37.743663 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.710087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmn9q\" (UniqueName: \"kubernetes.io/projected/9a15276c-947b-43db-abc2-d95c4025e505-kube-api-access-xmn9q\") pod \"ovnkube-node-m46hw\" (UID: \"9a15276c-947b-43db-abc2-d95c4025e505\") " pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-tmp-dir\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-system-cni-dir\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/baefe377-2257-4a6a-855b-231cd6cd313a-host-slash\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-host\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-system-cni-dir\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/baefe377-2257-4a6a-855b-231cd6cd313a-host-slash\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/baefe377-2257-4a6a-855b-231cd6cd313a-iptables-alerter-script\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-host\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.792843    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-cnibin\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.792907    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret podName:2ca195cd-f793-4606-b4d9-a8048bc5e102 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:38.292886274 +0000 UTC m=+3.094346069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret") pod "global-pull-secret-syncer-vlhsk" (UID: "2ca195cd-f793-4606-b4d9-a8048bc5e102") : object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-cnibin\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-cni-binary-copy\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-sys-fs\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.864307 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-socket-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ca195cd-f793-4606-b4d9-a8048bc5e102-dbus\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.792987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-tmp-dir\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hz4r\" (UniqueName: \"kubernetes.io/projected/baeece6e-85ae-48ff-8df3-50336c6cc138-kube-api-access-2hz4r\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793060    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-sys-fs\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-os-release\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-device-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-hosts-file\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8mkx\" (UniqueName: \"kubernetes.io/projected/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-kube-api-access-j8mkx\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ca195cd-f793-4606-b4d9-a8048bc5e102-dbus\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-device-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r7mw\" (UniqueName: \"kubernetes.io/projected/baefe377-2257-4a6a-855b-231cd6cd313a-kube-api-access-6r7mw\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ca195cd-f793-4606-b4d9-a8048bc5e102-kubelet-config\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-serviceca\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/baefe377-2257-4a6a-855b-231cd6cd313a-iptables-alerter-script\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-hosts-file\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:37.866423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-cni-binary-copy\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlmp9\" (UniqueName: \"kubernetes.io/projected/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-kube-api-access-vlmp9\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-socket-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-registration-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-tuning-conf-dir\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-registration-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-os-release\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ca195cd-f793-4606-b4d9-a8048bc5e102-kubelet-config\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/baeece6e-85ae-48ff-8df3-50336c6cc138-tuning-conf-dir\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw8bc\" (UniqueName: \"kubernetes.io/projected/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-kube-api-access-bw8bc\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.793981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.794095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-serviceca\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.867368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.794128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.794208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/baeece6e-85ae-48ff-8df3-50336c6cc138-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.801595    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.801614    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.801623    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rwtxq for pod openshift-network-diagnostics/network-check-target-l87mx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:37.801687    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq podName:35919494-e7be-4dd9-bc1f-3ac1127ab7f7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:38.301672294 +0000 UTC m=+3.103132085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rwtxq" (UniqueName: "kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq") pod "network-check-target-l87mx" (UID: "35919494-e7be-4dd9-bc1f-3ac1127ab7f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.804685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hz4r\" (UniqueName: \"kubernetes.io/projected/baeece6e-85ae-48ff-8df3-50336c6cc138-kube-api-access-2hz4r\") pod \"multus-additional-cni-plugins-47z5k\" (UID: \"baeece6e-85ae-48ff-8df3-50336c6cc138\") " pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.818818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlmp9\" (UniqueName: \"kubernetes.io/projected/6c7aa5bc-94bc-44db-956a-c7b126adcdf3-kube-api-access-vlmp9\") pod \"node-ca-g2qfq\" (UID: \"6c7aa5bc-94bc-44db-956a-c7b126adcdf3\") " pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.818862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw8bc\" (UniqueName: \"kubernetes.io/projected/5278bf56-5c67-4d5a-b39a-f1e85e42abd8-kube-api-access-bw8bc\") pod \"aws-ebs-csi-driver-node-l27tn\" (UID: \"5278bf56-5c67-4d5a-b39a-f1e85e42abd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.819353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r7mw\" (UniqueName: \"kubernetes.io/projected/baefe377-2257-4a6a-855b-231cd6cd313a-kube-api-access-6r7mw\") pod \"iptables-alerter-6cwzz\" (UID: \"baefe377-2257-4a6a-855b-231cd6cd313a\") " pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:37.868418 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.819690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8mkx\" (UniqueName: \"kubernetes.io/projected/6cdc1e12-fcd9-4084-903f-e4ea77df2fab-kube-api-access-j8mkx\") pod \"node-resolver-x5qct\" (UID: \"6cdc1e12-fcd9-4084-903f-e4ea77df2fab\") " pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.885382    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.899190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f9gnk"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.911027    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b8rmp"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.921568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.930599    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-47z5k"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.940631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6cwzz"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.953164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.958648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x5qct"
Jul 15 15:24:38.041997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:37.964438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g2qfq"
Jul 15 15:24:38.219035 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:38.197174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:38.219035 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.197344    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:38.219035 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.197418    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs podName:3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:39.197397867 +0000 UTC m=+3.998857902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs") pod "network-metrics-daemon-zkn9v" (UID: "3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:38.297738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.297928    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.298004    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret podName:2ca195cd-f793-4606-b4d9-a8048bc5e102 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:39.297983179 +0000 UTC m=+4.099442988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret") pod "global-pull-secret-syncer-vlhsk" (UID: "2ca195cd-f793-4606-b4d9-a8048bc5e102") : object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:38.398538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.398721    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.398744    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.398767    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rwtxq for pod openshift-network-diagnostics/network-check-target-l87mx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:38.430477 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:38.398838    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq podName:35919494-e7be-4dd9-bc1f-3ac1127ab7f7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:39.398810103 +0000 UTC m=+4.200269895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwtxq" (UniqueName: "kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq") pod "network-check-target-l87mx" (UID: "35919494-e7be-4dd9-bc1f-3ac1127ab7f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:38.623568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:38.623473    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 15:19:36 +0000 UTC" deadline="2027-12-30 13:00:24.837910786 +0000 UTC"
Jul 15 15:24:38.623568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:38.623523    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12789h35m46.214391259s"
Jul 15 15:24:38.982338 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:38.982298    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaefe377_2257_4a6a_855b_231cd6cd313a.slice/crio-9d65e99e6415ac462cc3687e074707926b61a05decaad42e7907a0239f31640a WatchSource:0}: Error finding container 9d65e99e6415ac462cc3687e074707926b61a05decaad42e7907a0239f31640a: Status 404 returned error can't find the container with id 9d65e99e6415ac462cc3687e074707926b61a05decaad42e7907a0239f31640a
Jul 15 15:24:38.985161 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:38.985077    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaeece6e_85ae_48ff_8df3_50336c6cc138.slice/crio-f9d6b50af76296028b4dd8b2f1938b2affbb27a4836b122c65f0969b49a06e03 WatchSource:0}: Error finding container f9d6b50af76296028b4dd8b2f1938b2affbb27a4836b122c65f0969b49a06e03: Status 404 returned error can't find the container with id f9d6b50af76296028b4dd8b2f1938b2affbb27a4836b122c65f0969b49a06e03
Jul 15 15:24:38.987377 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:38.987177    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a15276c_947b_43db_abc2_d95c4025e505.slice/crio-b35d7137015d83e9eb81abcfa2d960a242beddeb5954ded784de5d9c3f19e7f8 WatchSource:0}: Error finding container b35d7137015d83e9eb81abcfa2d960a242beddeb5954ded784de5d9c3f19e7f8: Status 404 returned error can't find the container with id b35d7137015d83e9eb81abcfa2d960a242beddeb5954ded784de5d9c3f19e7f8
Jul 15 15:24:38.988100 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:38.988076    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7aa5bc_94bc_44db_956a_c7b126adcdf3.slice/crio-27cd9d62f1d976a432fe35409302a12dec6e8f4a93b9844f55afcdf620022bfc WatchSource:0}: Error finding container 27cd9d62f1d976a432fe35409302a12dec6e8f4a93b9844f55afcdf620022bfc: Status 404 returned error can't find the container with id 27cd9d62f1d976a432fe35409302a12dec6e8f4a93b9844f55afcdf620022bfc
Jul 15 15:24:39.030477 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:24:39.030450    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cdc1e12_fcd9_4084_903f_e4ea77df2fab.slice/crio-ca67184e5791b9b63e14334e0f6d380c89a33be143332b67b1bdf45779a64b84 WatchSource:0}: Error finding container ca67184e5791b9b63e14334e0f6d380c89a33be143332b67b1bdf45779a64b84: Status 404 returned error can't find the container with id ca67184e5791b9b63e14334e0f6d380c89a33be143332b67b1bdf45779a64b84
Jul 15 15:24:39.204317 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.204171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:39.204317 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.204312    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:39.204470 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.204372    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs podName:3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:41.204358349 +0000 UTC m=+6.005818144 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs") pod "network-metrics-daemon-zkn9v" (UID: "3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:39.305279 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.305252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:39.305407 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.305357    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:39.305407 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.305405    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret podName:2ca195cd-f793-4606-b4d9-a8048bc5e102 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:41.305392724 +0000 UTC m=+6.106852519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret") pod "global-pull-secret-syncer-vlhsk" (UID: "2ca195cd-f793-4606-b4d9-a8048bc5e102") : object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:39.406689 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.406650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:39.406895 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.406854    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 15:24:39.406895 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.406875    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 15:24:39.406895 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.406893    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rwtxq for pod openshift-network-diagnostics/network-check-target-l87mx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:39.407060 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.406952    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq podName:35919494-e7be-4dd9-bc1f-3ac1127ab7f7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:41.406933214 +0000 UTC m=+6.208393024 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwtxq" (UniqueName: "kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq") pod "network-check-target-l87mx" (UID: "35919494-e7be-4dd9-bc1f-3ac1127ab7f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:39.762269 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.762235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:39.762452 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.762271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:39.762452 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.762388    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:39.762452 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.762428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:39.762595 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.762533    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:39.762653 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:39.762605    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:39.779617 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.779530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn" event={"ID":"5278bf56-5c67-4d5a-b39a-f1e85e42abd8","Type":"ContainerStarted","Data":"25b149efad9f346865b8a819a953c1443a18a734c8dc35a639cb7fde3d32c589"}
Jul 15 15:24:39.781885 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.781766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"b35d7137015d83e9eb81abcfa2d960a242beddeb5954ded784de5d9c3f19e7f8"}
Jul 15 15:24:39.784152 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.784087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerStarted","Data":"f9d6b50af76296028b4dd8b2f1938b2affbb27a4836b122c65f0969b49a06e03"}
Jul 15 15:24:39.786563 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.786499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6cwzz" event={"ID":"baefe377-2257-4a6a-855b-231cd6cd313a","Type":"ContainerStarted","Data":"9d65e99e6415ac462cc3687e074707926b61a05decaad42e7907a0239f31640a"}
Jul 15 15:24:39.793643 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.793053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal" event={"ID":"7b07821de4720a9cf3f16450e3c5031e","Type":"ContainerStarted","Data":"94beeec248fa5e6747fe47f059d67135ce234a2e63264bbc415b78b4fb167e30"}
Jul 15 15:24:39.799086 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.799016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x5qct" event={"ID":"6cdc1e12-fcd9-4084-903f-e4ea77df2fab","Type":"ContainerStarted","Data":"ca67184e5791b9b63e14334e0f6d380c89a33be143332b67b1bdf45779a64b84"}
Jul 15 15:24:39.800918 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.800800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9gnk" event={"ID":"1a3149f3-f5d7-40f0-921f-6f78f88b88ff","Type":"ContainerStarted","Data":"69b77583c772a3fad3efcba5033b9e5e0143fbdc609ad384a31f8ecdc2d524c2"}
Jul 15 15:24:39.804361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.804290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b8rmp" event={"ID":"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb","Type":"ContainerStarted","Data":"b0ed0c58aaa87dcd43cfe9bef89a87baf9e62d5ddb88dde0e498b1f54286f962"}
Jul 15 15:24:39.806121 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.806053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g2qfq" event={"ID":"6c7aa5bc-94bc-44db-956a-c7b126adcdf3","Type":"ContainerStarted","Data":"27cd9d62f1d976a432fe35409302a12dec6e8f4a93b9844f55afcdf620022bfc"}
Jul 15 15:24:39.809229 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.808821    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal" podStartSLOduration=3.808804471 podStartE2EDuration="3.808804471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:39.808237359 +0000 UTC m=+4.609697177" watchObservedRunningTime="2026-07-15 15:24:39.808804471 +0000 UTC m=+4.610264286"
Jul 15 15:24:39.815485 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.815457    2580 generic.go:356] "Generic (PLEG): container finished" podID="c335c55369566ce69dad6729899f194a" containerID="6959a76aac6a7c4b77051393f1c52bc2309d95bd73a5076471d57cc86a764086" exitCode=0
Jul 15 15:24:39.815607 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.815588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" event={"ID":"c335c55369566ce69dad6729899f194a","Type":"ContainerDied","Data":"6959a76aac6a7c4b77051393f1c52bc2309d95bd73a5076471d57cc86a764086"}
Jul 15 15:24:39.829497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:39.829465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mwv45" event={"ID":"30334585-bcd0-44b9-bdf0-0677a905c124","Type":"ContainerStarted","Data":"9a7aedaaddcf9b92c62435313b249d9ddf4a4c8ff4fda8324fb93d68b0995b75"}
Jul 15 15:24:40.845973 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:40.845905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" event={"ID":"c335c55369566ce69dad6729899f194a","Type":"ContainerStarted","Data":"5ad4c8f78797788e2e5441cf32952b534275e6aa2cfe88a973913751f4e26dff"}
Jul 15 15:24:40.869768 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:40.866860    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" podStartSLOduration=4.866839534 podStartE2EDuration="4.866839534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:40.865591352 +0000 UTC m=+5.667051190" watchObservedRunningTime="2026-07-15 15:24:40.866839534 +0000 UTC m=+5.668299352"
Jul 15 15:24:41.223632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:41.222979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:41.223632 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.223135    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:41.223632 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.223196    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs podName:3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:45.223178341 +0000 UTC m=+10.024638138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs") pod "network-metrics-daemon-zkn9v" (UID: "3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:41.324331 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:41.324295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:41.324563 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.324485    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:41.324650 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.324625    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret podName:2ca195cd-f793-4606-b4d9-a8048bc5e102 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:45.3246073 +0000 UTC m=+10.126067096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret") pod "global-pull-secret-syncer-vlhsk" (UID: "2ca195cd-f793-4606-b4d9-a8048bc5e102") : object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:41.425160 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:41.425104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:41.425404 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.425333    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 15:24:41.425404 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.425355    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 15:24:41.425404 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.425366    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rwtxq for pod openshift-network-diagnostics/network-check-target-l87mx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:41.425557 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.425425    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq podName:35919494-e7be-4dd9-bc1f-3ac1127ab7f7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:45.425406898 +0000 UTC m=+10.226866713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwtxq" (UniqueName: "kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq") pod "network-check-target-l87mx" (UID: "35919494-e7be-4dd9-bc1f-3ac1127ab7f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:41.763925 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:41.763891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:41.764145 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.764053    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:41.764145 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:41.764142    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:41.764269 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.764228    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:41.764320 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:41.764283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:41.764405 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:41.764383    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:43.762430 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:43.761847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:43.762430 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:43.761847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:43.762430 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:43.761993    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:43.762430 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:43.762095    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:43.762430 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:43.762191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:43.762430 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:43.762265    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:45.256834 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:45.256793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:45.257303 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.256948    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:45.257303 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.257013    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs podName:3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:53.256993983 +0000 UTC m=+18.058453798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs") pod "network-metrics-daemon-zkn9v" (UID: "3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:45.358149 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:45.358112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:45.358318 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.358255    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:45.358318 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.358314    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret podName:2ca195cd-f793-4606-b4d9-a8048bc5e102 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:53.358298357 +0000 UTC m=+18.159758149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret") pod "global-pull-secret-syncer-vlhsk" (UID: "2ca195cd-f793-4606-b4d9-a8048bc5e102") : object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:45.458505 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:45.458472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:45.458672 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.458624    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 15:24:45.458672 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.458651    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 15:24:45.458672 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.458665    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rwtxq for pod openshift-network-diagnostics/network-check-target-l87mx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:45.458866 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.458727    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq podName:35919494-e7be-4dd9-bc1f-3ac1127ab7f7 nodeName:}" failed. No retries permitted until 2026-07-15 15:24:53.458712405 +0000 UTC m=+18.260172210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwtxq" (UniqueName: "kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq") pod "network-check-target-l87mx" (UID: "35919494-e7be-4dd9-bc1f-3ac1127ab7f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:45.763252 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:45.762343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:45.763252 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.762498    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:45.763252 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:45.762959    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:45.763252 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.763054    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:45.763252 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:45.763112    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:45.763252 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:45.763209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:47.761207 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:47.761173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:47.761661 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:47.761318    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:47.761661 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:47.761503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:47.761661 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:47.761503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:47.761661 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:47.761592    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:47.761926 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:47.761671    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:49.762253 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:49.762169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:49.762253 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:49.762185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:49.762253 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:49.762170    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:49.762813 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:49.762349    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:49.762813 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:49.762424    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:49.762813 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:49.762505    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:51.761859 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:51.761824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:51.762315 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:51.761870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:51.762315 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:51.761948    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:51.762315 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:51.761824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:51.762315 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:51.762052    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:51.762315 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:51.762141    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:53.319811 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:53.319714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:53.320285 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.319869    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:53.320285 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.319953    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs podName:3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:09.319925205 +0000 UTC m=+34.121385014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs") pod "network-metrics-daemon-zkn9v" (UID: "3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:24:53.420700 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:53.420661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:53.420920 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.420844    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:53.420981 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.420922    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret podName:2ca195cd-f793-4606-b4d9-a8048bc5e102 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:09.420900467 +0000 UTC m=+34.222360261 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret") pod "global-pull-secret-syncer-vlhsk" (UID: "2ca195cd-f793-4606-b4d9-a8048bc5e102") : object "kube-system"/"original-pull-secret" not registered
Jul 15 15:24:53.521914 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:53.521873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:53.522062 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.522023    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 15:24:53.522062 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.522040    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 15:24:53.522062 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.522050    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rwtxq for pod openshift-network-diagnostics/network-check-target-l87mx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:53.522207 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.522108    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq podName:35919494-e7be-4dd9-bc1f-3ac1127ab7f7 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:09.522090675 +0000 UTC m=+34.323550483 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwtxq" (UniqueName: "kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq") pod "network-check-target-l87mx" (UID: "35919494-e7be-4dd9-bc1f-3ac1127ab7f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:24:53.761830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:53.761734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:53.761830 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:53.761781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:53.762047 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:53.761734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:53.762047 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.761879    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:53.762047 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.761962    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:53.762181 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:53.762071    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:55.764461 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:55.764431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:55.764872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:55.764435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:55.764872 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:55.764548    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:55.764872 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:55.764635    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:55.764872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:55.764436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:55.764872 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:55.764763    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:57.764338 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.764138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:57.764789 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:57.764463    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:57.764875 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.764855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:57.764979 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:57.764959    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:57.765105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.765012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:57.765186 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:57.765121    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:57.880477 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.880412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f9gnk" event={"ID":"1a3149f3-f5d7-40f0-921f-6f78f88b88ff","Type":"ContainerStarted","Data":"ff9f16ced8f9ffd0e7d24116db32b46358e00216bb12e1a3d8943b46e488c80e"}
Jul 15 15:24:57.882080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.882053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b8rmp" event={"ID":"6c10d3e8-a5ac-4ab7-b9c2-5b3c4965cbeb","Type":"ContainerStarted","Data":"4d6942faaf105b9b270df2ea029a298e045a54230edf657b9d38f677303e146a"}
Jul 15 15:24:57.883723 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.883675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g2qfq" event={"ID":"6c7aa5bc-94bc-44db-956a-c7b126adcdf3","Type":"ContainerStarted","Data":"da4f8b89e699a6c0a36db06c835e6ddcd9f566ca2b77861677c869c9d7858797"}
Jul 15 15:24:57.885624 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.885581    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mwv45" event={"ID":"30334585-bcd0-44b9-bdf0-0677a905c124","Type":"ContainerStarted","Data":"1c9c6e901bad2995ce60a28e14cae920155c06d51fe599e525b9054115ec4559"}
Jul 15 15:24:57.887324 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.887303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn" event={"ID":"5278bf56-5c67-4d5a-b39a-f1e85e42abd8","Type":"ContainerStarted","Data":"ac01e43d14cd4c67accef80d980231ec99b7702c35acce2a16c0a8fb6748ca4e"}
Jul 15 15:24:57.888695 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.888676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerStarted","Data":"e745f4a4df6650426c0d2c1243833901c4cf7ea301d832f5f4d90aebbc6e1f5b"}
Jul 15 15:24:57.898488 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.898435    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f9gnk" podStartSLOduration=4.230670001 podStartE2EDuration="22.898423854s" totalImagesPullingTime="18.667753853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:57.898235036 +0000 UTC m=+22.699694853" watchObservedRunningTime="2026-07-15 15:24:57.898423854 +0000 UTC m=+22.699883668"
Jul 15 15:24:57.919470 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.919426    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b8rmp" podStartSLOduration=4.261874148 podStartE2EDuration="22.919411633s" totalImagesPullingTime="18.657537485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:57.919160136 +0000 UTC m=+22.720619969" watchObservedRunningTime="2026-07-15 15:24:57.919411633 +0000 UTC m=+22.720871487"
Jul 15 15:24:57.936387 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.936337    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g2qfq" podStartSLOduration=3.295912279 podStartE2EDuration="21.936321896s" totalImagesPullingTime="18.640409617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:57.935799076 +0000 UTC m=+22.737258905" watchObservedRunningTime="2026-07-15 15:24:57.936321896 +0000 UTC m=+22.737781709"
Jul 15 15:24:57.953095 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:57.953050    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mwv45" podStartSLOduration=8.974396655 podStartE2EDuration="22.953035731s" totalImagesPullingTime="13.978639076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:57.952616599 +0000 UTC m=+22.754076423" watchObservedRunningTime="2026-07-15 15:24:57.953035731 +0000 UTC m=+22.754495555"
Jul 15 15:24:58.895040 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.894838    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:24:58.895861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.895410    2580 generic.go:356] "Generic (PLEG): container finished" podID="9a15276c-947b-43db-abc2-d95c4025e505" containerID="afcf1aa7fd424a43062b782ebe02c1d2bf46a50f28f55c7562010dab6f84e826" exitCode=1
Jul 15 15:24:58.895861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.895482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"c6aa44a3ec336043ae8ba4d0584cbbd9b7f898279713cc1dfcc7159c60a07d33"}
Jul 15 15:24:58.895861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.895609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"08a777c406a32eef019170780bb8c038fd9c2639d2ddc2aef0407d750faf4e39"}
Jul 15 15:24:58.895861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.895658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"757f39bb82b17356c84a91f0a49714a05c2da413745fd25c21bcacf6d5383708"}
Jul 15 15:24:58.895861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.895716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"cd0ec7edf0c64bb9987fe1284cd612e4406d51e67769a7ef362643f6ae5538ea"}
Jul 15 15:24:58.895861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.895765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerDied","Data":"afcf1aa7fd424a43062b782ebe02c1d2bf46a50f28f55c7562010dab6f84e826"}
Jul 15 15:24:58.895861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.895810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"09fd40002290ad0b46d726925758af6d4366776450138ff526ca67bf21effe8f"}
Jul 15 15:24:58.897070 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.897045    2580 generic.go:356] "Generic (PLEG): container finished" podID="baeece6e-85ae-48ff-8df3-50336c6cc138" containerID="e745f4a4df6650426c0d2c1243833901c4cf7ea301d832f5f4d90aebbc6e1f5b" exitCode=0
Jul 15 15:24:58.897179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.897137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerDied","Data":"e745f4a4df6650426c0d2c1243833901c4cf7ea301d832f5f4d90aebbc6e1f5b"}
Jul 15 15:24:58.898807 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.898768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x5qct" event={"ID":"6cdc1e12-fcd9-4084-903f-e4ea77df2fab","Type":"ContainerStarted","Data":"e27ace7c021897acad0af7ce7b38b9b36acfe5c012866464822bdaa4566784af"}
Jul 15 15:24:58.937279 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:58.937238    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-x5qct" podStartSLOduration=4.316583891 podStartE2EDuration="22.93722609s" totalImagesPullingTime="18.620642199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:58.937005329 +0000 UTC m=+23.738465144" watchObservedRunningTime="2026-07-15 15:24:58.93722609 +0000 UTC m=+23.738685914"
Jul 15 15:24:59.400993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.400969    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 15:24:59.660339 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.660193    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T15:24:59.400989112Z","UUID":"cf216aa6-b6da-4b69-b495-5fa2138db158","Handler":null,"Name":"","Endpoint":""}
Jul 15 15:24:59.661972 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.661816    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 15:24:59.662103 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.661995    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 15:24:59.761295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.761261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:24:59.761569 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.761261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:24:59.761569 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:59.761394    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:24:59.761569 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.761407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:24:59.761569 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:59.761554    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:24:59.761821 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:24:59.761627    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:24:59.902893 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.902862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn" event={"ID":"5278bf56-5c67-4d5a-b39a-f1e85e42abd8","Type":"ContainerStarted","Data":"dbeb365668a09ff3117d604df8e4bd7d91e553aa1153279e07717d650dbc495c"}
Jul 15 15:24:59.904577 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.904546    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6cwzz" event={"ID":"baefe377-2257-4a6a-855b-231cd6cd313a","Type":"ContainerStarted","Data":"2dc9cb707adb2667074ca3fabc93e4d3fdd7627817a3ce08a33bb6e796306969"}
Jul 15 15:24:59.921861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:24:59.921718    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6cwzz" podStartSLOduration=5.275257175 podStartE2EDuration="23.921699629s" totalImagesPullingTime="18.646442454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:24:59.920902621 +0000 UTC m=+24.722362438" watchObservedRunningTime="2026-07-15 15:24:59.921699629 +0000 UTC m=+24.723159461"
Jul 15 15:25:00.910645 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:00.910620    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:25:00.911150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:00.911037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"44fa858941146e32f84357b697c2a263cb0eb11c8156b5731d57eb4a1fc87852"}
Jul 15 15:25:01.137287 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.137211    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:25:01.137824 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.137799    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:25:01.761227 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.761194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:01.761385 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.761198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:01.761385 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:01.761332    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:25:01.761385 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.761197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:01.761385 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:01.761377    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:25:01.761526 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:01.761445    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:25:01.915342 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.915302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn" event={"ID":"5278bf56-5c67-4d5a-b39a-f1e85e42abd8","Type":"ContainerStarted","Data":"94d058c32afe2331182d480b8261ffc746951ff8177549b046a97127b4b51d4e"}
Jul 15 15:25:01.915957 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.915472    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:25:01.916140 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.916117    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mwv45"
Jul 15 15:25:01.934315 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:01.934265    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l27tn" podStartSLOduration=4.418557738 podStartE2EDuration="25.93424765s" totalImagesPullingTime="21.515689912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:01.932955269 +0000 UTC m=+26.734415082" watchObservedRunningTime="2026-07-15 15:25:01.93424765 +0000 UTC m=+26.735707465"
Jul 15 15:25:03.761693 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.761503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:03.762412 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.761503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:03.762412 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:03.761788    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:25:03.762412 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.761548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:03.762412 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:03.761910    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:25:03.762412 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:03.761848    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:25:03.923263 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.923236    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:25:03.923626 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.923583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"df6626eb125b2910945213783c41f95deb2b83542656983ada244d179b8ecfb0"}
Jul 15 15:25:03.923824 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.923809    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:25:03.924068 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.924044    2580 scope.go:117] "RemoveContainer" containerID="afcf1aa7fd424a43062b782ebe02c1d2bf46a50f28f55c7562010dab6f84e826"
Jul 15 15:25:03.927486 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.927020    2580 generic.go:356] "Generic (PLEG): container finished" podID="baeece6e-85ae-48ff-8df3-50336c6cc138" containerID="a37782dbcc433c0260005749eac94b28a8d961d4717df327a2b4dcec736c1ebc" exitCode=0
Jul 15 15:25:03.927486 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.927426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerDied","Data":"a37782dbcc433c0260005749eac94b28a8d961d4717df327a2b4dcec736c1ebc"}
Jul 15 15:25:03.941203 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:03.941185    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:25:04.932416 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.932266    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:25:04.932881 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.932794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" event={"ID":"9a15276c-947b-43db-abc2-d95c4025e505","Type":"ContainerStarted","Data":"4fb2239f97819d1dcbfc98e687cbb14b2176263ba87c0d9b6bc9cb720504f124"}
Jul 15 15:25:04.932998 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.932982    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:25:04.933163 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.933125    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:25:04.948035 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.948008    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:25:04.962515 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.962480    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw" podStartSLOduration=11.133926012 podStartE2EDuration="29.962469615s" totalImagesPullingTime="18.828543603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:04.961092327 +0000 UTC m=+29.762552140" watchObservedRunningTime="2026-07-15 15:25:04.962469615 +0000 UTC m=+29.763929429"
Jul 15 15:25:04.986943 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.986919    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlhsk"]
Jul 15 15:25:04.987040 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.987030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:04.987154 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:04.987134    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:25:04.990707 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.990683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l87mx"]
Jul 15 15:25:04.990819 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.990799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:04.990906 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:04.990887    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:25:04.991346 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.991317    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zkn9v"]
Jul 15 15:25:04.991440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:04.991412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:04.991506 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:04.991489    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:25:06.762021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:06.761995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:06.762534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:06.762095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:06.762534 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:06.762109    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:25:06.762534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:06.762122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:06.762534 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:06.762202    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:25:06.762534 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:06.762283    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:25:06.939069 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:06.939042    2580 generic.go:356] "Generic (PLEG): container finished" podID="baeece6e-85ae-48ff-8df3-50336c6cc138" containerID="f9198f786cda5cd355b02218fe8f38a96ec3273cdf31a2693a95f3785f333853" exitCode=0
Jul 15 15:25:06.939189 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:06.939129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerDied","Data":"f9198f786cda5cd355b02218fe8f38a96ec3273cdf31a2693a95f3785f333853"}
Jul 15 15:25:08.761319 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:08.761287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:08.761903 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:08.761287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:08.761903 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:08.761405    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87mx" podUID="35919494-e7be-4dd9-bc1f-3ac1127ab7f7"
Jul 15 15:25:08.761903 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:08.761288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:08.761903 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:08.761472    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkn9v" podUID="3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7"
Jul 15 15:25:08.761903 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:08.761575    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlhsk" podUID="2ca195cd-f793-4606-b4d9-a8048bc5e102"
Jul 15 15:25:09.146519 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:09.146359    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x5qct_6cdc1e12-fcd9-4084-903f-e4ea77df2fab/dns-node-resolver/0.log"
Jul 15 15:25:09.345188 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:09.345162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:09.345315 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.345264    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:25:09.345315 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.345310    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs podName:3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:41.34529762 +0000 UTC m=+66.146757417 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs") pod "network-metrics-daemon-zkn9v" (UID: "3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 15:25:09.446009 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:09.445939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:09.446130 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.446058    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 15:25:09.446130 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.446107    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret podName:2ca195cd-f793-4606-b4d9-a8048bc5e102 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:41.446093006 +0000 UTC m=+66.247552798 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret") pod "global-pull-secret-syncer-vlhsk" (UID: "2ca195cd-f793-4606-b4d9-a8048bc5e102") : object "kube-system"/"original-pull-secret" not registered
Jul 15 15:25:09.546637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:09.546606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:09.546815 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.546716    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 15:25:09.546815 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.546734    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 15:25:09.546815 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.546742    2580 projected.go:196] Error preparing data for projected volume kube-api-access-rwtxq for pod openshift-network-diagnostics/network-check-target-l87mx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:25:09.546815 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:09.546801    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq podName:35919494-e7be-4dd9-bc1f-3ac1127ab7f7 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:41.546788241 +0000 UTC m=+66.348248038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwtxq" (UniqueName: "kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq") pod "network-check-target-l87mx" (UID: "35919494-e7be-4dd9-bc1f-3ac1127ab7f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 15:25:09.947559 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:09.947524    2580 generic.go:356] "Generic (PLEG): container finished" podID="baeece6e-85ae-48ff-8df3-50336c6cc138" containerID="e5a1e17957e3d0e1d1083b28fe456e6544ef5c56aeacc3ca3d370609d0e032ba" exitCode=0
Jul 15 15:25:09.947905 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:09.947570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerDied","Data":"e5a1e17957e3d0e1d1083b28fe456e6544ef5c56aeacc3ca3d370609d0e032ba"}
Jul 15 15:25:10.114370 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.114345    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g2qfq_6c7aa5bc-94bc-44db-956a-c7b126adcdf3/node-ca/0.log"
Jul 15 15:25:10.441358 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.441265    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeReady"
Jul 15 15:25:10.441483 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.441371    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 15:25:10.761416 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.761328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:10.761416 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.761358    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:10.761416 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.761367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:10.764627 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.764606    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 15:25:10.764739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.764661    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qs6fl\""
Jul 15 15:25:10.764739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.764662    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 15:25:10.764867 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.764606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 15:25:10.765810 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.765794    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 15:25:10.765909 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:10.765887    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9ndf\""
Jul 15 15:25:14.960116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:14.960048    2580 generic.go:356] "Generic (PLEG): container finished" podID="baeece6e-85ae-48ff-8df3-50336c6cc138" containerID="0e3f55f925ec5bf192fc95934aef67d2045defc3ee03c8ac099bd0ede8b14b99" exitCode=0
Jul 15 15:25:14.960116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:14.960095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerDied","Data":"0e3f55f925ec5bf192fc95934aef67d2045defc3ee03c8ac099bd0ede8b14b99"}
Jul 15 15:25:15.964616 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:15.964581    2580 generic.go:356] "Generic (PLEG): container finished" podID="baeece6e-85ae-48ff-8df3-50336c6cc138" containerID="c0b593bf318618f3a5a6e7713adaa5f7d7565594a764836b88d5d0d0fc57bbe5" exitCode=0
Jul 15 15:25:15.964983 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:15.964635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerDied","Data":"c0b593bf318618f3a5a6e7713adaa5f7d7565594a764836b88d5d0d0fc57bbe5"}
Jul 15 15:25:16.973036 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:16.973005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-47z5k" event={"ID":"baeece6e-85ae-48ff-8df3-50336c6cc138","Type":"ContainerStarted","Data":"a6ecfe8582ae0928f7b5ebc9f17ff699eec0fd04bfc7e530bf524b64c5c9cc3c"}
Jul 15 15:25:17.004563 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:17.004520    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-47z5k" podStartSLOduration=8.502727289 podStartE2EDuration="41.004507688s" totalImagesPullingTime="32.501780399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:17.003032789 +0000 UTC m=+41.804492603" watchObservedRunningTime="2026-07-15 15:25:17.004507688 +0000 UTC m=+41.805967516"
Jul 15 15:25:29.115360 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.115323    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rk8x5"]
Jul 15 15:25:29.131125 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.131098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rrkcw"]
Jul 15 15:25:29.131248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.131228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.135578 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.135558    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 15:25:29.135578 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.135566    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 15:25:29.135909 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.135895    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fj5z9\""
Jul 15 15:25:29.144665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.144645    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d866f7d78-dnd5t"]
Jul 15 15:25:29.144788 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.144771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.147342 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.147325    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 15:25:29.147474 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.147459    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 15:25:29.147542 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.147464    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-scxjf\""
Jul 15 15:25:29.147542 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.147510    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 15:25:29.166735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.166686    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rk8x5"]
Jul 15 15:25:29.166735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.166716    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrkcw"]
Jul 15 15:25:29.166735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.166734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d866f7d78-dnd5t"]
Jul 15 15:25:29.166943 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.166832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.169565 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.169545    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 15:25:29.169955 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.169940    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 15:25:29.170049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.170020    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 15:25:29.170183 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.170168    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xdzpm\""
Jul 15 15:25:29.174398 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.174378    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 15:25:29.216747 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.216725    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5pmqj"]
Jul 15 15:25:29.238891 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.238874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5pmqj"]
Jul 15 15:25:29.238993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.238960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.241804 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.241785    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 15:25:29.241913 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.241819    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 15:25:29.241913 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.241822    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 15:25:29.241913 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.241880    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 15:25:29.242051 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.241993    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-92ld7\""
Jul 15 15:25:29.273268 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z6mr\" (UniqueName: \"kubernetes.io/projected/06e689da-a2a5-4e5c-84e4-51b9327d778c-kube-api-access-2z6mr\") pod \"ingress-canary-rrkcw\" (UID: \"06e689da-a2a5-4e5c-84e4-51b9327d778c\") " pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.273350 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d26eda-8254-44a1-853c-5a5cd65b16e0-tmp-dir\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.273350 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0ed422c-9464-4d54-8062-4f53fcce1489-registry-certificates\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273350 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e689da-a2a5-4e5c-84e4-51b9327d778c-cert\") pod \"ingress-canary-rrkcw\" (UID: \"06e689da-a2a5-4e5c-84e4-51b9327d778c\") " pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.273448 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-registry-tls\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273448 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr97s\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-kube-api-access-nr97s\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273448 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57cht\" (UniqueName: \"kubernetes.io/projected/47d26eda-8254-44a1-853c-5a5cd65b16e0-kube-api-access-57cht\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.273448 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0ed422c-9464-4d54-8062-4f53fcce1489-installation-pull-secrets\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-bound-sa-token\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0ed422c-9464-4d54-8062-4f53fcce1489-image-registry-private-configuration\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0ed422c-9464-4d54-8062-4f53fcce1489-trusted-ca\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273556    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47d26eda-8254-44a1-853c-5a5cd65b16e0-config-volume\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.273682 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0ed422c-9464-4d54-8062-4f53fcce1489-ca-trust-extracted\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.273682 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.273599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47d26eda-8254-44a1-853c-5a5cd65b16e0-metrics-tls\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.374939 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.374864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z6mr\" (UniqueName: \"kubernetes.io/projected/06e689da-a2a5-4e5c-84e4-51b9327d778c-kube-api-access-2z6mr\") pod \"ingress-canary-rrkcw\" (UID: \"06e689da-a2a5-4e5c-84e4-51b9327d778c\") " pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.374939 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.374896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d26eda-8254-44a1-853c-5a5cd65b16e0-tmp-dir\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.374939 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.374914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0ed422c-9464-4d54-8062-4f53fcce1489-registry-certificates\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.374939 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.374933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e83d2d43-3886-433f-a47c-cef5d46c90f7-data-volume\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.374977    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e689da-a2a5-4e5c-84e4-51b9327d778c-cert\") pod \"ingress-canary-rrkcw\" (UID: \"06e689da-a2a5-4e5c-84e4-51b9327d778c\") " pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e83d2d43-3886-433f-a47c-cef5d46c90f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-registry-tls\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr97s\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-kube-api-access-nr97s\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57cht\" (UniqueName: \"kubernetes.io/projected/47d26eda-8254-44a1-853c-5a5cd65b16e0-kube-api-access-57cht\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0ed422c-9464-4d54-8062-4f53fcce1489-installation-pull-secrets\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-bound-sa-token\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.375236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0ed422c-9464-4d54-8062-4f53fcce1489-image-registry-private-configuration\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0ed422c-9464-4d54-8062-4f53fcce1489-trusted-ca\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47d26eda-8254-44a1-853c-5a5cd65b16e0-config-volume\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d26eda-8254-44a1-853c-5a5cd65b16e0-tmp-dir\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcjdg\" (UniqueName: \"kubernetes.io/projected/e83d2d43-3886-433f-a47c-cef5d46c90f7-kube-api-access-gcjdg\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0ed422c-9464-4d54-8062-4f53fcce1489-ca-trust-extracted\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e83d2d43-3886-433f-a47c-cef5d46c90f7-crio-socket\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47d26eda-8254-44a1-853c-5a5cd65b16e0-metrics-tls\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.375598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e83d2d43-3886-433f-a47c-cef5d46c90f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.375991 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.375845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0ed422c-9464-4d54-8062-4f53fcce1489-registry-certificates\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.376048 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.376007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0ed422c-9464-4d54-8062-4f53fcce1489-ca-trust-extracted\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.376480 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.376329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47d26eda-8254-44a1-853c-5a5cd65b16e0-config-volume\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.376480 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.376434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0ed422c-9464-4d54-8062-4f53fcce1489-trusted-ca\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.379348 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.379318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47d26eda-8254-44a1-853c-5a5cd65b16e0-metrics-tls\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.379452 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.379392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0ed422c-9464-4d54-8062-4f53fcce1489-installation-pull-secrets\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.379452 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.379401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e689da-a2a5-4e5c-84e4-51b9327d778c-cert\") pod \"ingress-canary-rrkcw\" (UID: \"06e689da-a2a5-4e5c-84e4-51b9327d778c\") " pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.379452 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.379421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0ed422c-9464-4d54-8062-4f53fcce1489-image-registry-private-configuration\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.379627 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.379604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-registry-tls\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.390352 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.390326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z6mr\" (UniqueName: \"kubernetes.io/projected/06e689da-a2a5-4e5c-84e4-51b9327d778c-kube-api-access-2z6mr\") pod \"ingress-canary-rrkcw\" (UID: \"06e689da-a2a5-4e5c-84e4-51b9327d778c\") " pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.390447 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.390404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57cht\" (UniqueName: \"kubernetes.io/projected/47d26eda-8254-44a1-853c-5a5cd65b16e0-kube-api-access-57cht\") pod \"dns-default-rk8x5\" (UID: \"47d26eda-8254-44a1-853c-5a5cd65b16e0\") " pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.391203 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.391183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr97s\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-kube-api-access-nr97s\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.391309 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.391292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0ed422c-9464-4d54-8062-4f53fcce1489-bound-sa-token\") pod \"image-registry-5d866f7d78-dnd5t\" (UID: \"f0ed422c-9464-4d54-8062-4f53fcce1489\") " pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.442183 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.442150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:29.452832 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.452812    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrkcw"
Jul 15 15:25:29.475554 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.475527    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:29.475822 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.475803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e83d2d43-3886-433f-a47c-cef5d46c90f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.475933 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.475914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjdg\" (UniqueName: \"kubernetes.io/projected/e83d2d43-3886-433f-a47c-cef5d46c90f7-kube-api-access-gcjdg\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.475974 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.475953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e83d2d43-3886-433f-a47c-cef5d46c90f7-crio-socket\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.476024 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.475991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e83d2d43-3886-433f-a47c-cef5d46c90f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.476060 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.476043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e83d2d43-3886-433f-a47c-cef5d46c90f7-data-volume\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.476108 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.476067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e83d2d43-3886-433f-a47c-cef5d46c90f7-crio-socket\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.476372 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.476350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e83d2d43-3886-433f-a47c-cef5d46c90f7-data-volume\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.476465 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.476427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e83d2d43-3886-433f-a47c-cef5d46c90f7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.478490 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.478452    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e83d2d43-3886-433f-a47c-cef5d46c90f7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.488742 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.488721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjdg\" (UniqueName: \"kubernetes.io/projected/e83d2d43-3886-433f-a47c-cef5d46c90f7-kube-api-access-gcjdg\") pod \"insights-runtime-extractor-5pmqj\" (UID: \"e83d2d43-3886-433f-a47c-cef5d46c90f7\") " pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.546651 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.546624    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5pmqj"
Jul 15 15:25:29.678102 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.672329    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rk8x5"]
Jul 15 15:25:29.693473 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:29.693441    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d26eda_8254_44a1_853c_5a5cd65b16e0.slice/crio-2b546a6f86c2ed50d32351da5f2eff72e70b12812734d4a38de203f8b206cd3c WatchSource:0}: Error finding container 2b546a6f86c2ed50d32351da5f2eff72e70b12812734d4a38de203f8b206cd3c: Status 404 returned error can't find the container with id 2b546a6f86c2ed50d32351da5f2eff72e70b12812734d4a38de203f8b206cd3c
Jul 15 15:25:29.751934 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.751899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5pmqj"]
Jul 15 15:25:29.755306 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:29.755280    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83d2d43_3886_433f_a47c_cef5d46c90f7.slice/crio-1af4070095f22421743faa70d0f50d66508dc15744f1aff11d0b32b53daed6d0 WatchSource:0}: Error finding container 1af4070095f22421743faa70d0f50d66508dc15744f1aff11d0b32b53daed6d0: Status 404 returned error can't find the container with id 1af4070095f22421743faa70d0f50d66508dc15744f1aff11d0b32b53daed6d0
Jul 15 15:25:29.881387 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.881356    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrkcw"]
Jul 15 15:25:29.884736 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:29.884713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d866f7d78-dnd5t"]
Jul 15 15:25:29.884858 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:29.884742    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e689da_a2a5_4e5c_84e4_51b9327d778c.slice/crio-04e4f0fac53287ab57d6e4707324a0de71adabec4bdcd1ac2088aa1e46dfd121 WatchSource:0}: Error finding container 04e4f0fac53287ab57d6e4707324a0de71adabec4bdcd1ac2088aa1e46dfd121: Status 404 returned error can't find the container with id 04e4f0fac53287ab57d6e4707324a0de71adabec4bdcd1ac2088aa1e46dfd121
Jul 15 15:25:29.887223 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:29.887206    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ed422c_9464_4d54_8062_4f53fcce1489.slice/crio-4caef23b0f71f9418175044b3e1608da6a598bc02323cc8b85f41a121f5205f8 WatchSource:0}: Error finding container 4caef23b0f71f9418175044b3e1608da6a598bc02323cc8b85f41a121f5205f8: Status 404 returned error can't find the container with id 4caef23b0f71f9418175044b3e1608da6a598bc02323cc8b85f41a121f5205f8
Jul 15 15:25:30.003534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:30.003500    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5pmqj" event={"ID":"e83d2d43-3886-433f-a47c-cef5d46c90f7","Type":"ContainerStarted","Data":"1af4070095f22421743faa70d0f50d66508dc15744f1aff11d0b32b53daed6d0"}
Jul 15 15:25:30.004546 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:30.004514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t" event={"ID":"f0ed422c-9464-4d54-8062-4f53fcce1489","Type":"ContainerStarted","Data":"4caef23b0f71f9418175044b3e1608da6a598bc02323cc8b85f41a121f5205f8"}
Jul 15 15:25:30.005540 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:30.005519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rk8x5" event={"ID":"47d26eda-8254-44a1-853c-5a5cd65b16e0","Type":"ContainerStarted","Data":"2b546a6f86c2ed50d32351da5f2eff72e70b12812734d4a38de203f8b206cd3c"}
Jul 15 15:25:30.006391 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:30.006370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrkcw" event={"ID":"06e689da-a2a5-4e5c-84e4-51b9327d778c","Type":"ContainerStarted","Data":"04e4f0fac53287ab57d6e4707324a0de71adabec4bdcd1ac2088aa1e46dfd121"}
Jul 15 15:25:31.011302 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:31.011094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t" event={"ID":"f0ed422c-9464-4d54-8062-4f53fcce1489","Type":"ContainerStarted","Data":"83c736c3f05428733ba03894791005ad053d7e6780fa6f37344f3cceea39f13a"}
Jul 15 15:25:31.011302 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:31.011219    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:31.034381 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:31.034239    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t" podStartSLOduration=2.034222123 podStartE2EDuration="2.034222123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:31.032618819 +0000 UTC m=+55.834078677" watchObservedRunningTime="2026-07-15 15:25:31.034222123 +0000 UTC m=+55.835681936"
Jul 15 15:25:34.393871 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:34.393843    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"]
Jul 15 15:25:34.412323 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:34.412288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"]
Jul 15 15:25:34.412474 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:34.412412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:34.420069 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:34.420036    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7bchc\""
Jul 15 15:25:34.420309 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:34.420292    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 15:25:34.526007 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:34.525975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0977e6dc-43b0-4918-a5af-6c47be1cf669-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7lg2h\" (UID: \"0977e6dc-43b0-4918-a5af-6c47be1cf669\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:34.627462 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:34.627419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0977e6dc-43b0-4918-a5af-6c47be1cf669-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7lg2h\" (UID: \"0977e6dc-43b0-4918-a5af-6c47be1cf669\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:34.627640 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:34.627593    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 15:25:34.627727 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:34.627713    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0977e6dc-43b0-4918-a5af-6c47be1cf669-tls-certificates podName:0977e6dc-43b0-4918-a5af-6c47be1cf669 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:35.127690191 +0000 UTC m=+59.929150005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/0977e6dc-43b0-4918-a5af-6c47be1cf669-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-7lg2h" (UID: "0977e6dc-43b0-4918-a5af-6c47be1cf669") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 15:25:35.131149 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.131106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0977e6dc-43b0-4918-a5af-6c47be1cf669-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7lg2h\" (UID: \"0977e6dc-43b0-4918-a5af-6c47be1cf669\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:35.133878 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.133846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0977e6dc-43b0-4918-a5af-6c47be1cf669-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7lg2h\" (UID: \"0977e6dc-43b0-4918-a5af-6c47be1cf669\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:35.324131 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.324095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:35.742716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.742678    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8547f4674-zklnj"]
Jul 15 15:25:35.745035 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.745012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:35.748617 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.748594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sqhvr\""
Jul 15 15:25:35.748736 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.748596    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 15:25:35.749431 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.749409    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 15:25:35.749528 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.749448    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 15:25:35.749528 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.749508    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 15:25:35.749643 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.749452    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 15:25:35.749805 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.749788    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 15:25:35.749934 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.749850    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 15:25:35.757855 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.757830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8547f4674-zklnj"]
Jul 15 15:25:35.937784 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.937732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-oauth-serving-cert\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:35.937984 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.937811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-console-config\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:35.937984 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.937843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtjw9\" (UniqueName: \"kubernetes.io/projected/064b7790-34f6-41af-8e5f-670dfeaae744-kube-api-access-xtjw9\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:35.938110 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.937982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-serving-cert\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:35.938110 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.938031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-oauth-config\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:35.938110 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:35.938094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-service-ca\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.041588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-console-config\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.041641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtjw9\" (UniqueName: \"kubernetes.io/projected/064b7790-34f6-41af-8e5f-670dfeaae744-kube-api-access-xtjw9\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.041767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-serving-cert\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.041791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-oauth-config\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.041818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-service-ca\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.041867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-oauth-serving-cert\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.042630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-oauth-serving-cert\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044174 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.043204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-console-config\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.044794 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.044572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-service-ca\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.047979 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.047936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-serving-cert\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.049539 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.049496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-oauth-config\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.052030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.052011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtjw9\" (UniqueName: \"kubernetes.io/projected/064b7790-34f6-41af-8e5f-670dfeaae744-kube-api-access-xtjw9\") pod \"console-8547f4674-zklnj\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") " pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.055032 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.055004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:36.109546 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.109515    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"]
Jul 15 15:25:36.115214 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:36.115174    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0977e6dc_43b0_4918_a5af_6c47be1cf669.slice/crio-df671c740a51bfe67133eccebce0a13ad619e5d3d9a873127ad7f35ce749ffec WatchSource:0}: Error finding container df671c740a51bfe67133eccebce0a13ad619e5d3d9a873127ad7f35ce749ffec: Status 404 returned error can't find the container with id df671c740a51bfe67133eccebce0a13ad619e5d3d9a873127ad7f35ce749ffec
Jul 15 15:25:36.209403 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.209334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8547f4674-zklnj"]
Jul 15 15:25:36.212176 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:36.212145    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod064b7790_34f6_41af_8e5f_670dfeaae744.slice/crio-6225992521d09939af4a35a6a79f4945ac43a95dde8b8883ed3a6eea91fc89bb WatchSource:0}: Error finding container 6225992521d09939af4a35a6a79f4945ac43a95dde8b8883ed3a6eea91fc89bb: Status 404 returned error can't find the container with id 6225992521d09939af4a35a6a79f4945ac43a95dde8b8883ed3a6eea91fc89bb
Jul 15 15:25:36.952792 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:36.952766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m46hw"
Jul 15 15:25:37.031901 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.031858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h" event={"ID":"0977e6dc-43b0-4918-a5af-6c47be1cf669","Type":"ContainerStarted","Data":"df671c740a51bfe67133eccebce0a13ad619e5d3d9a873127ad7f35ce749ffec"}
Jul 15 15:25:37.034263 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.034234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5pmqj" event={"ID":"e83d2d43-3886-433f-a47c-cef5d46c90f7","Type":"ContainerStarted","Data":"e41369b0dd0d1241cc69b12c8a2655c3761024cda86920241b19c5c121b3b642"}
Jul 15 15:25:37.036227 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.036179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8547f4674-zklnj" event={"ID":"064b7790-34f6-41af-8e5f-670dfeaae744","Type":"ContainerStarted","Data":"6225992521d09939af4a35a6a79f4945ac43a95dde8b8883ed3a6eea91fc89bb"}
Jul 15 15:25:37.038320 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.038289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rk8x5" event={"ID":"47d26eda-8254-44a1-853c-5a5cd65b16e0","Type":"ContainerStarted","Data":"37ea73e9a17307515ed9cde057d524c13d80c47117fe9d89b9ea97d03f89a813"}
Jul 15 15:25:37.038414 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.038337    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rk8x5" event={"ID":"47d26eda-8254-44a1-853c-5a5cd65b16e0","Type":"ContainerStarted","Data":"bc4c699aed7f86930370bc2fe6c8b7aac76b8cdd21328b5a5dd2a5145b7e8f9c"}
Jul 15 15:25:37.038808 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.038790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:37.040387 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.040362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrkcw" event={"ID":"06e689da-a2a5-4e5c-84e4-51b9327d778c","Type":"ContainerStarted","Data":"b4e002e45ad677f846e6153abf12efd1b7202bc64b4731e62238fcbc5ff050da"}
Jul 15 15:25:37.061732 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.061169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rk8x5" podStartSLOduration=1.7854500770000001 podStartE2EDuration="8.061149961s" totalImagesPullingTime="6.275699884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:37.061069796 +0000 UTC m=+61.862529611" watchObservedRunningTime="2026-07-15 15:25:37.061149961 +0000 UTC m=+61.862609795"
Jul 15 15:25:37.078413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:37.078360    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rrkcw" podStartSLOduration=1.9973959940000001 podStartE2EDuration="8.078343306s" totalImagesPullingTime="6.080947312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:37.077288341 +0000 UTC m=+61.878748157" watchObservedRunningTime="2026-07-15 15:25:37.078343306 +0000 UTC m=+61.879803102"
Jul 15 15:25:38.045991 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:38.045945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h" event={"ID":"0977e6dc-43b0-4918-a5af-6c47be1cf669","Type":"ContainerStarted","Data":"ae4e7fdf0ef4fd01a592d021bde635b8f9eb7f1587ab9e8bf74427c8d3c0b255"}
Jul 15 15:25:38.064584 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:38.064535    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h" podStartSLOduration=2.930218483 podStartE2EDuration="4.064521657s" totalImagesPullingTime="1.134303174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:38.063948486 +0000 UTC m=+62.865408302" watchObservedRunningTime="2026-07-15 15:25:38.064521657 +0000 UTC m=+62.865981471"
Jul 15 15:25:39.049739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.049703    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:39.055198 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.055172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7lg2h"
Jul 15 15:25:39.458713 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.458683    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-tpqjq"]
Jul 15 15:25:39.460997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.460974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.463776 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.463742    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 15:25:39.464850 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.464831    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 15:25:39.464960 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.464934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 15 15:25:39.465030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.464964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 15 15:25:39.465030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.464964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jgvfr\""
Jul 15 15:25:39.465130 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.465050    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 15:25:39.470109 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.470084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwj4q\" (UniqueName: \"kubernetes.io/projected/b3031f1f-e0ba-4095-a829-1d1c512ba618-kube-api-access-gwj4q\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.470213 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.470142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.470393 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.470231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.470393 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.470353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3031f1f-e0ba-4095-a829-1d1c512ba618-metrics-client-ca\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.471822 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.471803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-tpqjq"]
Jul 15 15:25:39.570846 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.570812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.571016 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.570900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3031f1f-e0ba-4095-a829-1d1c512ba618-metrics-client-ca\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.571016 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.570931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwj4q\" (UniqueName: \"kubernetes.io/projected/b3031f1f-e0ba-4095-a829-1d1c512ba618-kube-api-access-gwj4q\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.571016 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.570970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.571198 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:39.571069    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 15 15:25:39.571198 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:39.571135    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-tls podName:b3031f1f-e0ba-4095-a829-1d1c512ba618 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:40.071117328 +0000 UTC m=+64.872577140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-tls") pod "prometheus-operator-64785b6675-tpqjq" (UID: "b3031f1f-e0ba-4095-a829-1d1c512ba618") : secret "prometheus-operator-tls" not found
Jul 15 15:25:39.571990 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.571956    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3031f1f-e0ba-4095-a829-1d1c512ba618-metrics-client-ca\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.573637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.573612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:39.582509 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:39.582485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwj4q\" (UniqueName: \"kubernetes.io/projected/b3031f1f-e0ba-4095-a829-1d1c512ba618-kube-api-access-gwj4q\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:40.074743 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:40.074706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:40.077057 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:40.077030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3031f1f-e0ba-4095-a829-1d1c512ba618-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-tpqjq\" (UID: \"b3031f1f-e0ba-4095-a829-1d1c512ba618\") " pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:40.371882 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:40.371801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq"
Jul 15 15:25:40.846248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:40.846134    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-tpqjq"]
Jul 15 15:25:40.848931 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:40.848892    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3031f1f_e0ba_4095_a829_1d1c512ba618.slice/crio-edde50eb0f2d2d4b385bcc13c4aad210e5c60fbd518d04df945ee8b6f1207100 WatchSource:0}: Error finding container edde50eb0f2d2d4b385bcc13c4aad210e5c60fbd518d04df945ee8b6f1207100: Status 404 returned error can't find the container with id edde50eb0f2d2d4b385bcc13c4aad210e5c60fbd518d04df945ee8b6f1207100
Jul 15 15:25:41.056933 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.056846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq" event={"ID":"b3031f1f-e0ba-4095-a829-1d1c512ba618","Type":"ContainerStarted","Data":"edde50eb0f2d2d4b385bcc13c4aad210e5c60fbd518d04df945ee8b6f1207100"}
Jul 15 15:25:41.058530 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.058502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5pmqj" event={"ID":"e83d2d43-3886-433f-a47c-cef5d46c90f7","Type":"ContainerStarted","Data":"5ccc33f5df0f8d2f3684f55ac2ca3007fa53046b836552648681d2edbc85534c"}
Jul 15 15:25:41.059920 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.059895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8547f4674-zklnj" event={"ID":"064b7790-34f6-41af-8e5f-670dfeaae744","Type":"ContainerStarted","Data":"9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd"}
Jul 15 15:25:41.078696 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.078661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8547f4674-zklnj" podStartSLOduration=1.564356211 podStartE2EDuration="6.078648884s" totalImagesPullingTime="4.514292673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:41.078612054 +0000 UTC m=+65.880071881" watchObservedRunningTime="2026-07-15 15:25:41.078648884 +0000 UTC m=+65.880108699"
Jul 15 15:25:41.386411 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.386371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:41.389424 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.389399    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 15:25:41.400189 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.400158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7-metrics-certs\") pod \"network-metrics-daemon-zkn9v\" (UID: \"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7\") " pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:41.487539 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.487507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:41.490962 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.490937    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 15:25:41.501049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.501025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ca195cd-f793-4606-b4d9-a8048bc5e102-original-pull-secret\") pod \"global-pull-secret-syncer-vlhsk\" (UID: \"2ca195cd-f793-4606-b4d9-a8048bc5e102\") " pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:41.588258 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.588213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:41.591366 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.591336    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 15:25:41.601296 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.601276    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 15:25:41.612267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.612241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwtxq\" (UniqueName: \"kubernetes.io/projected/35919494-e7be-4dd9-bc1f-3ac1127ab7f7-kube-api-access-rwtxq\") pod \"network-check-target-l87mx\" (UID: \"35919494-e7be-4dd9-bc1f-3ac1127ab7f7\") " pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:41.674843 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.674774    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qs6fl\""
Jul 15 15:25:41.680162 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.680133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9ndf\""
Jul 15 15:25:41.682034 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.682011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlhsk"
Jul 15 15:25:41.683140 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.683120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkn9v"
Jul 15 15:25:41.687893 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.687872    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:41.861339 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:41.861265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlhsk"]
Jul 15 15:25:42.090223 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:42.090191    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l87mx"]
Jul 15 15:25:42.097285 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:42.097205    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zkn9v"]
Jul 15 15:25:42.151325 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:42.151287    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca195cd_f793_4606_b4d9_a8048bc5e102.slice/crio-632362f41be2411281e562ae19c6c9ea6fd0fae2bf9cb006cc00668930937f99 WatchSource:0}: Error finding container 632362f41be2411281e562ae19c6c9ea6fd0fae2bf9cb006cc00668930937f99: Status 404 returned error can't find the container with id 632362f41be2411281e562ae19c6c9ea6fd0fae2bf9cb006cc00668930937f99
Jul 15 15:25:42.152746 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:42.152701    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35919494_e7be_4dd9_bc1f_3ac1127ab7f7.slice/crio-5e9cf5373960e1f065cb6266e5933ed4778664dec6308b3a6389534cb9d753f7 WatchSource:0}: Error finding container 5e9cf5373960e1f065cb6266e5933ed4778664dec6308b3a6389534cb9d753f7: Status 404 returned error can't find the container with id 5e9cf5373960e1f065cb6266e5933ed4778664dec6308b3a6389534cb9d753f7
Jul 15 15:25:42.153687 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:42.153658    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3853faa6_00bb_4f61_bbc5_e2a5e3c9f4e7.slice/crio-fd37e95f0970c24fdbdc4656ec75fc0f883e0c99e5e4ee7a11052d07450c2c94 WatchSource:0}: Error finding container fd37e95f0970c24fdbdc4656ec75fc0f883e0c99e5e4ee7a11052d07450c2c94: Status 404 returned error can't find the container with id fd37e95f0970c24fdbdc4656ec75fc0f883e0c99e5e4ee7a11052d07450c2c94
Jul 15 15:25:43.069910 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.069872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkn9v" event={"ID":"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7","Type":"ContainerStarted","Data":"fd37e95f0970c24fdbdc4656ec75fc0f883e0c99e5e4ee7a11052d07450c2c94"}
Jul 15 15:25:43.072121 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.072076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l87mx" event={"ID":"35919494-e7be-4dd9-bc1f-3ac1127ab7f7","Type":"ContainerStarted","Data":"5e9cf5373960e1f065cb6266e5933ed4778664dec6308b3a6389534cb9d753f7"}
Jul 15 15:25:43.075486 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.074972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq" event={"ID":"b3031f1f-e0ba-4095-a829-1d1c512ba618","Type":"ContainerStarted","Data":"d7125415fba5b1c93a4129f096d1a0b32dd353c8129b2582d376816182b20e82"}
Jul 15 15:25:43.075486 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.075024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq" event={"ID":"b3031f1f-e0ba-4095-a829-1d1c512ba618","Type":"ContainerStarted","Data":"60801bf7bcdfb73e6b45ce1b55f3ee2bfae92ac1dc24168d49c6b59e902b07b2"}
Jul 15 15:25:43.076506 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.076449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlhsk" event={"ID":"2ca195cd-f793-4606-b4d9-a8048bc5e102","Type":"ContainerStarted","Data":"632362f41be2411281e562ae19c6c9ea6fd0fae2bf9cb006cc00668930937f99"}
Jul 15 15:25:43.096046 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.095982    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-tpqjq" podStartSLOduration=2.76593099 podStartE2EDuration="4.095963437s" totalImagesPullingTime="1.330032447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:43.095314837 +0000 UTC m=+67.896774676" watchObservedRunningTime="2026-07-15 15:25:43.095963437 +0000 UTC m=+67.897423232"
Jul 15 15:25:43.521604 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.521061    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56c754b4d5-c8n6l"]
Jul 15 15:25:43.528005 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.527966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.536164 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.536139    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c754b4d5-c8n6l"]
Jul 15 15:25:43.538729 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.538539    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 15:25:43.605392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.605061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ttng\" (UniqueName: \"kubernetes.io/projected/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-kube-api-access-6ttng\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.605392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.605124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-serving-cert\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.605392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.605150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-oauth-config\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.605392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.605202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-oauth-serving-cert\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.605392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.605231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-config\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.605392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.605266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-service-ca\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.605392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.605295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-trusted-ca-bundle\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.706173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.705996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-oauth-serving-cert\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.706173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.706046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-config\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.706173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.706076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-service-ca\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.706173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.706104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-trusted-ca-bundle\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.706173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.706130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ttng\" (UniqueName: \"kubernetes.io/projected/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-kube-api-access-6ttng\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.706173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.706170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-serving-cert\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.706580 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.706194    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-oauth-config\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.707150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.707095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-config\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.707150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.707136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-oauth-serving-cert\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.707329 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.707232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-service-ca\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.707329 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.707260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-trusted-ca-bundle\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.710337 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.710315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-serving-cert\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.711367 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.711338    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-oauth-config\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.717443 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.717409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ttng\" (UniqueName: \"kubernetes.io/projected/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-kube-api-access-6ttng\") pod \"console-56c754b4d5-c8n6l\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") " pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:43.840001 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:43.839960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:44.981804 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:44.981252    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"]
Jul 15 15:25:45.012785 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.012734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"]
Jul 15 15:25:45.012945 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.012804    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r78gp"]
Jul 15 15:25:45.013001 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.012981    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.016032 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.015996    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 15 15:25:45.016429 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.016406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 15 15:25:45.016700 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.016682    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5crnq\""
Jul 15 15:25:45.026589 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.026556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.026589 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.026561    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"]
Jul 15 15:25:45.030162 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.030138    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vstdl\""
Jul 15 15:25:45.030299 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.030138    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 15:25:45.030408 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.030368    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 15:25:45.030408 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.030381    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 15:25:45.036210 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.036191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.038842 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.038810    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 15 15:25:45.038939 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.038870    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 15 15:25:45.040160 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.040139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-897s8\""
Jul 15 15:25:45.040255 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.040146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 15 15:25:45.050200 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.050177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"]
Jul 15 15:25:45.117787 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.117739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmnnv\" (UniqueName: \"kubernetes.io/projected/ed72d9d4-8ef9-4b37-986c-b66253087631-kube-api-access-bmnnv\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.117971 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.117813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.117971 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.117871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed72d9d4-8ef9-4b37-986c-b66253087631-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.117971 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.117894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-sys\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.117971 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.117922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed72d9d4-8ef9-4b37-986c-b66253087631-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.117971 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.117945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-root\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.118206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.117996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.118206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-tls\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.118206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.118206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-wtmp\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.118206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.118206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdkd4\" (UniqueName: \"kubernetes.io/projected/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-api-access-jdkd4\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.118206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.118469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed72d9d4-8ef9-4b37-986c-b66253087631-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.118469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1dcacea-7f36-47c3-9e55-935a6320fc76-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.118469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jvqw\" (UniqueName: \"kubernetes.io/projected/67c16a39-6b90-49af-8513-497d3f8e49b7-kube-api-access-9jvqw\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.118469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f1dcacea-7f36-47c3-9e55-935a6320fc76-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.118469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-textfile\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.118469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.118401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c16a39-6b90-49af-8513-497d3f8e49b7-metrics-client-ca\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.219628 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.219628 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed72d9d4-8ef9-4b37-986c-b66253087631-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.219889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1dcacea-7f36-47c3-9e55-935a6320fc76-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.219889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jvqw\" (UniqueName: \"kubernetes.io/projected/67c16a39-6b90-49af-8513-497d3f8e49b7-kube-api-access-9jvqw\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.219889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f1dcacea-7f36-47c3-9e55-935a6320fc76-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.219889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-textfile\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.219889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c16a39-6b90-49af-8513-497d3f8e49b7-metrics-client-ca\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.219889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmnnv\" (UniqueName: \"kubernetes.io/projected/ed72d9d4-8ef9-4b37-986c-b66253087631-kube-api-access-bmnnv\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.219889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed72d9d4-8ef9-4b37-986c-b66253087631-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-sys\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed72d9d4-8ef9-4b37-986c-b66253087631-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.219973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-root\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-tls\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-wtmp\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdkd4\" (UniqueName: \"kubernetes.io/projected/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-api-access-jdkd4\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.220734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f1dcacea-7f36-47c3-9e55-935a6320fc76-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.220734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1dcacea-7f36-47c3-9e55-935a6320fc76-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.220734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-root\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220512    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.220734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-wtmp\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.220734 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:45.220115    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 15 15:25:45.220734 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:45.220686    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-tls podName:f1dcacea-7f36-47c3-9e55-935a6320fc76 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:45.720668142 +0000 UTC m=+70.522127942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-qnbsh" (UID: "f1dcacea-7f36-47c3-9e55-935a6320fc76") : secret "kube-state-metrics-tls" not found
Jul 15 15:25:45.221114 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-textfile\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.221114 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed72d9d4-8ef9-4b37-986c-b66253087631-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.221114 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.220903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c16a39-6b90-49af-8513-497d3f8e49b7-sys\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.221265 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:45.221217    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 15:25:45.221326 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:45.221276    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-tls podName:67c16a39-6b90-49af-8513-497d3f8e49b7 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:45.721258515 +0000 UTC m=+70.522718327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-tls") pod "node-exporter-r78gp" (UID: "67c16a39-6b90-49af-8513-497d3f8e49b7") : secret "node-exporter-tls" not found
Jul 15 15:25:45.221378 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.221352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67c16a39-6b90-49af-8513-497d3f8e49b7-metrics-client-ca\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.221378 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.221362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-accelerators-collector-config\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.222573 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.222548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed72d9d4-8ef9-4b37-986c-b66253087631-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.223105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.223078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.223641 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.223615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.223724 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.223693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed72d9d4-8ef9-4b37-986c-b66253087631-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.254336 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.254281    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jvqw\" (UniqueName: \"kubernetes.io/projected/67c16a39-6b90-49af-8513-497d3f8e49b7-kube-api-access-9jvqw\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.254696 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.254677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmnnv\" (UniqueName: \"kubernetes.io/projected/ed72d9d4-8ef9-4b37-986c-b66253087631-kube-api-access-bmnnv\") pod \"openshift-state-metrics-84bd68b896-wvvjn\" (UID: \"ed72d9d4-8ef9-4b37-986c-b66253087631\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.258395 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.258375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdkd4\" (UniqueName: \"kubernetes.io/projected/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-api-access-jdkd4\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.324570 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.324523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"
Jul 15 15:25:45.725309 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.725275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.725491 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.725360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-tls\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.728519 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.728442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67c16a39-6b90-49af-8513-497d3f8e49b7-node-exporter-tls\") pod \"node-exporter-r78gp\" (UID: \"67c16a39-6b90-49af-8513-497d3f8e49b7\") " pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.728636 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.728542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1dcacea-7f36-47c3-9e55-935a6320fc76-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-qnbsh\" (UID: \"f1dcacea-7f36-47c3-9e55-935a6320fc76\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:45.935684 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.935646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r78gp"
Jul 15 15:25:45.946744 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:45.946724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"
Jul 15 15:25:46.055926 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.055900    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:46.056259 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.055982    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:46.060727 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.060708    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:46.085146 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.085120    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:25:46.098636 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.098608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:25:46.098817 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.098801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.101254 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101234    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 15:25:46.101355 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101274    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 15:25:46.101355 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101322    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 15:25:46.101445 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101408    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 15:25:46.101591 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 15:25:46.101645 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101630    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 15:25:46.101813 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101776    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 15:25:46.101954 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101831    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 15:25:46.101954 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.101840    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 15:25:46.102109 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.102092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rcwpr\""
Jul 15 15:25:46.111944 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.111924    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:25:46.129878 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.129853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.129898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.129961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130111 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130111 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130111 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130111 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw2p9\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-kube-api-access-jw2p9\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130266 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130266 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130363 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130363 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130451 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.130502 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.130454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231212 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231212 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231212 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw2p9\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-kube-api-access-jw2p9\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231975 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231975 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.231975 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:46.231626    2580 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 15 15:25:46.231975 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:46.231710    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls podName:ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e nodeName:}" failed. No retries permitted until 2026-07-15 15:25:46.731686374 +0000 UTC m=+71.533146183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e") : secret "alertmanager-main-tls" not found
Jul 15 15:25:46.231975 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.231902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.232791 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.232493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.232791 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.232689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.234621 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.234587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.235520 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.235490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.235665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.235645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.236004 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.235981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.236046 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.235976    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.236141 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.236081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.237153 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.237129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.237853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.237828    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.252338 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.252314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw2p9\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-kube-api-access-jw2p9\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.734562 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.734530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:46.737147 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:46.737119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:47.008267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:47.008181    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:25:47.049061 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:47.049026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rk8x5"
Jul 15 15:25:47.759462 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:47.759304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c754b4d5-c8n6l"]
Jul 15 15:25:47.861014 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:47.860941    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c7a16a_3085_4623_a9f5_4f7ab3dc6925.slice/crio-95ea227e87c90894a0250ae7aae9459726c46fa7409e8b7d2fbc3cac881f253d WatchSource:0}: Error finding container 95ea227e87c90894a0250ae7aae9459726c46fa7409e8b7d2fbc3cac881f253d: Status 404 returned error can't find the container with id 95ea227e87c90894a0250ae7aae9459726c46fa7409e8b7d2fbc3cac881f253d
Jul 15 15:25:47.867519 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:47.867320    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c16a39_6b90_49af_8513_497d3f8e49b7.slice/crio-4976b70f7a46298666711ce7350a4eed1b25f7fbc95e3771082ed33b5dbd0aad WatchSource:0}: Error finding container 4976b70f7a46298666711ce7350a4eed1b25f7fbc95e3771082ed33b5dbd0aad: Status 404 returned error can't find the container with id 4976b70f7a46298666711ce7350a4eed1b25f7fbc95e3771082ed33b5dbd0aad
Jul 15 15:25:48.080471 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.080393    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn"]
Jul 15 15:25:48.084211 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:48.084162    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded72d9d4_8ef9_4b37_986c_b66253087631.slice/crio-b4e30dfb35bac36042887c97a77a3f7a5ca7c07aea48e8ec78f6fb80e53ddebb WatchSource:0}: Error finding container b4e30dfb35bac36042887c97a77a3f7a5ca7c07aea48e8ec78f6fb80e53ddebb: Status 404 returned error can't find the container with id b4e30dfb35bac36042887c97a77a3f7a5ca7c07aea48e8ec78f6fb80e53ddebb
Jul 15 15:25:48.098250 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.098221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r78gp" event={"ID":"67c16a39-6b90-49af-8513-497d3f8e49b7","Type":"ContainerStarted","Data":"4976b70f7a46298666711ce7350a4eed1b25f7fbc95e3771082ed33b5dbd0aad"}
Jul 15 15:25:48.100723 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.100692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn" event={"ID":"ed72d9d4-8ef9-4b37-986c-b66253087631","Type":"ContainerStarted","Data":"b4e30dfb35bac36042887c97a77a3f7a5ca7c07aea48e8ec78f6fb80e53ddebb"}
Jul 15 15:25:48.103511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.103348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c754b4d5-c8n6l" event={"ID":"71c7a16a-3085-4623-a9f5-4f7ab3dc6925","Type":"ContainerStarted","Data":"4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e"}
Jul 15 15:25:48.103511 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.103400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c754b4d5-c8n6l" event={"ID":"71c7a16a-3085-4623-a9f5-4f7ab3dc6925","Type":"ContainerStarted","Data":"95ea227e87c90894a0250ae7aae9459726c46fa7409e8b7d2fbc3cac881f253d"}
Jul 15 15:25:48.110050 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.110007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l87mx" event={"ID":"35919494-e7be-4dd9-bc1f-3ac1127ab7f7","Type":"ContainerStarted","Data":"6cc81de1c8b9519aa84ba009aeffc5048612d1714e3d464a7600ef576d61c4ac"}
Jul 15 15:25:48.114050 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.113992    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:25:48.116598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.116501    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5pmqj" event={"ID":"e83d2d43-3886-433f-a47c-cef5d46c90f7","Type":"ContainerStarted","Data":"05210e6f21b1735025834d7b12a38e9de16117390d749eb3e2a745f52879c7e8"}
Jul 15 15:25:48.128402 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.128380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-qnbsh"]
Jul 15 15:25:48.135532 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:48.135510    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1dcacea_7f36_47c3_9e55_935a6320fc76.slice/crio-ee9697888b6cc10d55e7b188807d1526a56b8b2c094c02cb194fadb424c6087b WatchSource:0}: Error finding container ee9697888b6cc10d55e7b188807d1526a56b8b2c094c02cb194fadb424c6087b: Status 404 returned error can't find the container with id ee9697888b6cc10d55e7b188807d1526a56b8b2c094c02cb194fadb424c6087b
Jul 15 15:25:48.147463 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.146099    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56c754b4d5-c8n6l" podStartSLOduration=5.146078451 podStartE2EDuration="5.146078451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:48.132040853 +0000 UTC m=+72.933500668" watchObservedRunningTime="2026-07-15 15:25:48.146078451 +0000 UTC m=+72.947538266"
Jul 15 15:25:48.147463 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.147027    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:25:48.160625 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.160580    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l87mx" podStartSLOduration=67.446349529 podStartE2EDuration="1m13.160560811s" totalImagesPullingTime="5.714211282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:48.158462573 +0000 UTC m=+72.959922408" watchObservedRunningTime="2026-07-15 15:25:48.160560811 +0000 UTC m=+72.962020603"
Jul 15 15:25:48.164419 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:48.164392    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceaac3d0_c0dc_4213_a37b_e53d1fd44d3e.slice/crio-11bba41cce4869c78f3a793dfc4235b249776666b46c96ec65e42705a7afa3c2 WatchSource:0}: Error finding container 11bba41cce4869c78f3a793dfc4235b249776666b46c96ec65e42705a7afa3c2: Status 404 returned error can't find the container with id 11bba41cce4869c78f3a793dfc4235b249776666b46c96ec65e42705a7afa3c2
Jul 15 15:25:48.180330 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.180248    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5pmqj" podStartSLOduration=1.458732689 podStartE2EDuration="19.18023188s" totalImagesPullingTime="17.721499191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:25:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:48.178823033 +0000 UTC m=+72.980282848" watchObservedRunningTime="2026-07-15 15:25:48.18023188 +0000 UTC m=+72.981691708"
Jul 15 15:25:48.952357 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.952328    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-864774d8b5-ztztl"]
Jul 15 15:25:48.955051 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.955022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:48.958419 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.957988    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 15:25:48.958419 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.958044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 15:25:48.958419 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.958085    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 15:25:48.958419 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.957988    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 15:25:48.958419 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.958189    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 15:25:48.958419 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.958319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c1r26sl5dfh8j\""
Jul 15 15:25:48.959305 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.959164    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vrzvc\""
Jul 15 15:25:48.968082 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:48.968063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-864774d8b5-ztztl"]
Jul 15 15:25:49.064171 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.064335 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.064335 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76jtk\" (UniqueName: \"kubernetes.io/projected/e3ee76aa-7994-48ee-b164-186cdf3ecff1-kube-api-access-76jtk\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.064335 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.064335 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-tls\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.064335 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.064593 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-grpc-tls\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.064593 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.064410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee76aa-7994-48ee-b164-186cdf3ecff1-metrics-client-ca\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.123775 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.123730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlhsk" event={"ID":"2ca195cd-f793-4606-b4d9-a8048bc5e102","Type":"ContainerStarted","Data":"1815ad2589afc6a7940b7a0ce1356068fbd5c51488b6c331cfff82c2fcf71f6c"}
Jul 15 15:25:49.127070 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.127041    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn" event={"ID":"ed72d9d4-8ef9-4b37-986c-b66253087631","Type":"ContainerStarted","Data":"eb414d9a335ddc25352cbafcd6c52c4199c80335a8cb574d6f429753dbfba60d"}
Jul 15 15:25:49.127286 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.127262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn" event={"ID":"ed72d9d4-8ef9-4b37-986c-b66253087631","Type":"ContainerStarted","Data":"b232caba9f9a96ad0feed55088c0bf0184bf0b61e2ddd97a7ffd7c9da467b832"}
Jul 15 15:25:49.129551 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.129342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkn9v" event={"ID":"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7","Type":"ContainerStarted","Data":"1cd98a12b1e3d72cf9e58d39d065dfdd662e6a64e61efb6ad36572fe8c76a4e7"}
Jul 15 15:25:49.129551 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.129389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkn9v" event={"ID":"3853faa6-00bb-4f61-bbc5-e2a5e3c9f4e7","Type":"ContainerStarted","Data":"87960f4609daf8cd8dfd4b9654955917fee16efcf1d14371225a3472206947d0"}
Jul 15 15:25:49.130802 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.130779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerStarted","Data":"11bba41cce4869c78f3a793dfc4235b249776666b46c96ec65e42705a7afa3c2"}
Jul 15 15:25:49.132342 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.132284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh" event={"ID":"f1dcacea-7f36-47c3-9e55-935a6320fc76","Type":"ContainerStarted","Data":"ee9697888b6cc10d55e7b188807d1526a56b8b2c094c02cb194fadb424c6087b"}
Jul 15 15:25:49.143799 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.143746    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vlhsk" podStartSLOduration=67.427311658 podStartE2EDuration="1m13.143728545s" totalImagesPullingTime="5.716416887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 15:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:49.142352038 +0000 UTC m=+73.943811880" watchObservedRunningTime="2026-07-15 15:25:49.143728545 +0000 UTC m=+73.945188361"
Jul 15 15:25:49.160702 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.160660    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zkn9v" podStartSLOduration=68.420556963 podStartE2EDuration="1m14.160647134s" totalImagesPullingTime="5.740090171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:49.159597616 +0000 UTC m=+73.961057431" watchObservedRunningTime="2026-07-15 15:25:49.160647134 +0000 UTC m=+73.962106947"
Jul 15 15:25:49.164837 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.164798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.164950 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.164869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76jtk\" (UniqueName: \"kubernetes.io/projected/e3ee76aa-7994-48ee-b164-186cdf3ecff1-kube-api-access-76jtk\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.165008 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.164959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.165067 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.165013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-tls\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.165067 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.165043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.165163 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.165077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-grpc-tls\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.165163 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.165108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee76aa-7994-48ee-b164-186cdf3ecff1-metrics-client-ca\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.165251 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.165227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.167731 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.167653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee76aa-7994-48ee-b164-186cdf3ecff1-metrics-client-ca\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.169119 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.169075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.169962 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.169918    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.170225 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.170069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.170937 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.170913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-tls\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.171011 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.170987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.171089 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.171068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3ee76aa-7994-48ee-b164-186cdf3ecff1-secret-grpc-tls\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.174894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.174856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76jtk\" (UniqueName: \"kubernetes.io/projected/e3ee76aa-7994-48ee-b164-186cdf3ecff1-kube-api-access-76jtk\") pod \"thanos-querier-864774d8b5-ztztl\" (UID: \"e3ee76aa-7994-48ee-b164-186cdf3ecff1\") " pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.267989 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.267904    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:49.473477 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.473440    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-774f6f965c-45gpn"]
Jul 15 15:25:49.475807 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.475785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.481251 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.481231    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 15:25:49.481363 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.481254    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 15 15:25:49.482406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.482376    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-34hotute99ffl\""
Jul 15 15:25:49.482406 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.482403    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-678qq\""
Jul 15 15:25:49.482553 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.482516    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 15 15:25:49.482871 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.482854    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 15 15:25:49.516105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.516078    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-774f6f965c-45gpn"]
Jul 15 15:25:49.568999 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.568596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a961e48d-8f70-4e29-9b66-b997e3f96c7c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.568999 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.568645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a961e48d-8f70-4e29-9b66-b997e3f96c7c-audit-log\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.568999 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.568686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-secret-metrics-server-tls\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.568999 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.568718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a961e48d-8f70-4e29-9b66-b997e3f96c7c-metrics-server-audit-profiles\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.568999 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.568786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-client-ca-bundle\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.568999 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.568827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgbmd\" (UniqueName: \"kubernetes.io/projected/a961e48d-8f70-4e29-9b66-b997e3f96c7c-kube-api-access-qgbmd\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.568999 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.568880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-secret-metrics-server-client-certs\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.655157 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.655128    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"]
Jul 15 15:25:49.659494 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.659467    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:49.662203 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.662171    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 15 15:25:49.662993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.662652    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dqbfq\""
Jul 15 15:25:49.670010 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.669918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a961e48d-8f70-4e29-9b66-b997e3f96c7c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670010 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.669962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a961e48d-8f70-4e29-9b66-b997e3f96c7c-audit-log\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670677 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.670219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-secret-metrics-server-tls\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670677 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.670265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a961e48d-8f70-4e29-9b66-b997e3f96c7c-metrics-server-audit-profiles\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670677 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.670315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a961e48d-8f70-4e29-9b66-b997e3f96c7c-audit-log\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670677 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.670318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-client-ca-bundle\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670960 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.670718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgbmd\" (UniqueName: \"kubernetes.io/projected/a961e48d-8f70-4e29-9b66-b997e3f96c7c-kube-api-access-qgbmd\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670960 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.670812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-secret-metrics-server-client-certs\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.670960 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.670940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a961e48d-8f70-4e29-9b66-b997e3f96c7c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.671931 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.671905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a961e48d-8f70-4e29-9b66-b997e3f96c7c-metrics-server-audit-profiles\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.672037 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.671921    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"]
Jul 15 15:25:49.673581 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.673540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-client-ca-bundle\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.673744 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.673675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-secret-metrics-server-client-certs\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.673872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.673829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a961e48d-8f70-4e29-9b66-b997e3f96c7c-secret-metrics-server-tls\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.685703 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.685669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgbmd\" (UniqueName: \"kubernetes.io/projected/a961e48d-8f70-4e29-9b66-b997e3f96c7c-kube-api-access-qgbmd\") pod \"metrics-server-774f6f965c-45gpn\" (UID: \"a961e48d-8f70-4e29-9b66-b997e3f96c7c\") " pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.771767 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.771724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffdff59c-f9ad-4301-bfae-21a665db1481-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-25kfx\" (UID: \"ffdff59c-f9ad-4301-bfae-21a665db1481\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:49.771894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.771801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm7qz\" (UniqueName: \"kubernetes.io/projected/ffdff59c-f9ad-4301-bfae-21a665db1481-kube-api-access-mm7qz\") pod \"monitoring-plugin-5f68df7f58-25kfx\" (UID: \"ffdff59c-f9ad-4301-bfae-21a665db1481\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:49.788179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.788039    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:25:49.873274 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.873222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffdff59c-f9ad-4301-bfae-21a665db1481-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-25kfx\" (UID: \"ffdff59c-f9ad-4301-bfae-21a665db1481\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:49.873404 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.873285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm7qz\" (UniqueName: \"kubernetes.io/projected/ffdff59c-f9ad-4301-bfae-21a665db1481-kube-api-access-mm7qz\") pod \"monitoring-plugin-5f68df7f58-25kfx\" (UID: \"ffdff59c-f9ad-4301-bfae-21a665db1481\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:49.873470 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:49.873410    2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 15 15:25:49.873521 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:25:49.873481    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffdff59c-f9ad-4301-bfae-21a665db1481-monitoring-plugin-cert podName:ffdff59c-f9ad-4301-bfae-21a665db1481 nodeName:}" failed. No retries permitted until 2026-07-15 15:25:50.37346009 +0000 UTC m=+75.174919884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/ffdff59c-f9ad-4301-bfae-21a665db1481-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-25kfx" (UID: "ffdff59c-f9ad-4301-bfae-21a665db1481") : secret "monitoring-plugin-cert" not found
Jul 15 15:25:49.884698 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.884669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm7qz\" (UniqueName: \"kubernetes.io/projected/ffdff59c-f9ad-4301-bfae-21a665db1481-kube-api-access-mm7qz\") pod \"monitoring-plugin-5f68df7f58-25kfx\" (UID: \"ffdff59c-f9ad-4301-bfae-21a665db1481\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:49.915087 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.914985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-864774d8b5-ztztl"]
Jul 15 15:25:49.976827 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:49.976800    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-774f6f965c-45gpn"]
Jul 15 15:25:49.981544 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:49.981516    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda961e48d_8f70_4e29_9b66_b997e3f96c7c.slice/crio-e07d478d562d8c850aae339c3060a1ebbe31714c37f97295cdb68f8ea74ade34 WatchSource:0}: Error finding container e07d478d562d8c850aae339c3060a1ebbe31714c37f97295cdb68f8ea74ade34: Status 404 returned error can't find the container with id e07d478d562d8c850aae339c3060a1ebbe31714c37f97295cdb68f8ea74ade34
Jul 15 15:25:50.137671 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.137583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn" event={"ID":"a961e48d-8f70-4e29-9b66-b997e3f96c7c","Type":"ContainerStarted","Data":"e07d478d562d8c850aae339c3060a1ebbe31714c37f97295cdb68f8ea74ade34"}
Jul 15 15:25:50.138797 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.138776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" event={"ID":"e3ee76aa-7994-48ee-b164-186cdf3ecff1","Type":"ContainerStarted","Data":"badadebb6e5def8f01869220b2e46c2b347be63a11599a56038dfea136c3a702"}
Jul 15 15:25:50.140913 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.140891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn" event={"ID":"ed72d9d4-8ef9-4b37-986c-b66253087631","Type":"ContainerStarted","Data":"0d4944269df3b0bef721c3547cd2c71adf046ad89c6862354c486debb4bdd55a"}
Jul 15 15:25:50.142992 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.142972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh" event={"ID":"f1dcacea-7f36-47c3-9e55-935a6320fc76","Type":"ContainerStarted","Data":"2f4238976709847db8708b293a45a65a62a59f1f2a995a592484297cfdb91152"}
Jul 15 15:25:50.143096 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.143014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh" event={"ID":"f1dcacea-7f36-47c3-9e55-935a6320fc76","Type":"ContainerStarted","Data":"d581bbd126629cb9c15e29d813a4fd9868739c9947bd6e7a176354be0c0b56b9"}
Jul 15 15:25:50.143096 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.143036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh" event={"ID":"f1dcacea-7f36-47c3-9e55-935a6320fc76","Type":"ContainerStarted","Data":"8ed0f386037eb1340a90dabf4dc5eebb81e173cbf58c1eede391de6e0050a6bc"}
Jul 15 15:25:50.167395 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.167343    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wvvjn" podStartSLOduration=4.653301618 podStartE2EDuration="6.167324804s" totalImagesPullingTime="1.514023186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:50.165966582 +0000 UTC m=+74.967426397" watchObservedRunningTime="2026-07-15 15:25:50.167324804 +0000 UTC m=+74.968784623"
Jul 15 15:25:50.187131 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.187074    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-qnbsh" podStartSLOduration=4.575069935 podStartE2EDuration="6.187057157s" totalImagesPullingTime="1.611987222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:50.186102858 +0000 UTC m=+74.987562694" watchObservedRunningTime="2026-07-15 15:25:50.187057157 +0000 UTC m=+74.988516972"
Jul 15 15:25:50.379907 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.379872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffdff59c-f9ad-4301-bfae-21a665db1481-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-25kfx\" (UID: \"ffdff59c-f9ad-4301-bfae-21a665db1481\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:50.382206 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.382182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffdff59c-f9ad-4301-bfae-21a665db1481-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-25kfx\" (UID: \"ffdff59c-f9ad-4301-bfae-21a665db1481\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:50.571049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.571016    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:50.744793 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.744420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"]
Jul 15 15:25:50.748458 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:50.748433    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffdff59c_f9ad_4301_bfae_21a665db1481.slice/crio-42289dbee3d61130b22ed35afa840123ac6df64a9058deae52e6e879a3d58052 WatchSource:0}: Error finding container 42289dbee3d61130b22ed35afa840123ac6df64a9058deae52e6e879a3d58052: Status 404 returned error can't find the container with id 42289dbee3d61130b22ed35afa840123ac6df64a9058deae52e6e879a3d58052
Jul 15 15:25:50.956042 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.956007    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56c754b4d5-c8n6l"]
Jul 15 15:25:50.993849 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.993821    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5ffc5689d4-9swcx"]
Jul 15 15:25:50.998642 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:50.998460    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.013503 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.013476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ffc5689d4-9swcx"]
Jul 15 15:25:51.087214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.087175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-serving-cert\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.087364 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.087223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-oauth-config\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.087364 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.087315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-service-ca\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.087364 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.087359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpj4b\" (UniqueName: \"kubernetes.io/projected/a763b861-2044-4f24-9ed0-801cc90ba91d-kube-api-access-xpj4b\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.087482 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.087387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-console-config\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.087482 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.087469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-trusted-ca-bundle\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.087584 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.087505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-oauth-serving-cert\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.151447 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.151237    2580 generic.go:356] "Generic (PLEG): container finished" podID="67c16a39-6b90-49af-8513-497d3f8e49b7" containerID="69793fa91b305145d43cbb2ba2743c69396405e04247db4ec754b1c3bfefead0" exitCode=0
Jul 15 15:25:51.151447 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.151292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r78gp" event={"ID":"67c16a39-6b90-49af-8513-497d3f8e49b7","Type":"ContainerDied","Data":"69793fa91b305145d43cbb2ba2743c69396405e04247db4ec754b1c3bfefead0"}
Jul 15 15:25:51.153181 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.153153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx" event={"ID":"ffdff59c-f9ad-4301-bfae-21a665db1481","Type":"ContainerStarted","Data":"42289dbee3d61130b22ed35afa840123ac6df64a9058deae52e6e879a3d58052"}
Jul 15 15:25:51.189231 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.189077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-serving-cert\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.189231 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.189117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-oauth-config\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.189440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.189285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-service-ca\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.189440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.189387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpj4b\" (UniqueName: \"kubernetes.io/projected/a763b861-2044-4f24-9ed0-801cc90ba91d-kube-api-access-xpj4b\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.189440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.189420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-console-config\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.190052 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.189725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-trusted-ca-bundle\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.190052 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.189802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-oauth-serving-cert\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.190203 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.190099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-console-config\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.190837 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.190651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-trusted-ca-bundle\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.190837 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.190795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-service-ca\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.191519 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.191331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-oauth-serving-cert\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.192412 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.192360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-serving-cert\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.193151 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.193109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-oauth-config\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.201698 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.201675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpj4b\" (UniqueName: \"kubernetes.io/projected/a763b861-2044-4f24-9ed0-801cc90ba91d-kube-api-access-xpj4b\") pod \"console-5ffc5689d4-9swcx\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") " pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.311712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.311670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:25:51.867384 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:51.867243    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ffc5689d4-9swcx"]
Jul 15 15:25:51.894805 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:25:51.894742    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda763b861_2044_4f24_9ed0_801cc90ba91d.slice/crio-8c5910bc28b60f4361cacae15b0a167ea5d9c4fcd38103bb2f6b6267c4201936 WatchSource:0}: Error finding container 8c5910bc28b60f4361cacae15b0a167ea5d9c4fcd38103bb2f6b6267c4201936: Status 404 returned error can't find the container with id 8c5910bc28b60f4361cacae15b0a167ea5d9c4fcd38103bb2f6b6267c4201936
Jul 15 15:25:52.020943 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.020849    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d866f7d78-dnd5t"
Jul 15 15:25:52.158112 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.158070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn" event={"ID":"a961e48d-8f70-4e29-9b66-b997e3f96c7c","Type":"ContainerStarted","Data":"9126c534fbf62a512e470401e1929cae18a5169ff6a43f490c2d616c3f819d08"}
Jul 15 15:25:52.160684 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.160653    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r78gp" event={"ID":"67c16a39-6b90-49af-8513-497d3f8e49b7","Type":"ContainerStarted","Data":"e672a0e0322d25a2c4a997aaed41d94a04396aa72a22161aa1f2fb9b17952f57"}
Jul 15 15:25:52.160827 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.160711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r78gp" event={"ID":"67c16a39-6b90-49af-8513-497d3f8e49b7","Type":"ContainerStarted","Data":"211bce47490b369097ba37e0489a2a9b2976845460695f8401e85fc7e74a523a"}
Jul 15 15:25:52.162415 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.162386    2580 generic.go:356] "Generic (PLEG): container finished" podID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerID="dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1" exitCode=0
Jul 15 15:25:52.162559 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.162520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1"}
Jul 15 15:25:52.164317 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.164267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ffc5689d4-9swcx" event={"ID":"a763b861-2044-4f24-9ed0-801cc90ba91d","Type":"ContainerStarted","Data":"0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90"}
Jul 15 15:25:52.164317 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.164306    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ffc5689d4-9swcx" event={"ID":"a763b861-2044-4f24-9ed0-801cc90ba91d","Type":"ContainerStarted","Data":"8c5910bc28b60f4361cacae15b0a167ea5d9c4fcd38103bb2f6b6267c4201936"}
Jul 15 15:25:52.182623 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.182573    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn" podStartSLOduration=1.472842867 podStartE2EDuration="3.182554234s" totalImagesPullingTime="1.709711367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:52.181347963 +0000 UTC m=+76.982807779" watchObservedRunningTime="2026-07-15 15:25:52.182554234 +0000 UTC m=+76.984014044"
Jul 15 15:25:52.205567 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.205462    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r78gp" podStartSLOduration=5.352736574 podStartE2EDuration="8.205443254s" totalImagesPullingTime="2.776314898s" totalInitContainerRuntime="76.391782ms" isStatefulPod=true podCreationTimestamp="2026-07-15 15:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:52.203090613 +0000 UTC m=+77.004550430" watchObservedRunningTime="2026-07-15 15:25:52.205443254 +0000 UTC m=+77.006903070"
Jul 15 15:25:52.258219 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:52.258155    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5ffc5689d4-9swcx" podStartSLOduration=2.258133925 podStartE2EDuration="2.258133925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:52.22668973 +0000 UTC m=+77.028149545" watchObservedRunningTime="2026-07-15 15:25:52.258133925 +0000 UTC m=+77.059593740"
Jul 15 15:25:53.176300 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:53.176254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx" event={"ID":"ffdff59c-f9ad-4301-bfae-21a665db1481","Type":"ContainerStarted","Data":"ee3c0267a86f57ef840b1408a57a308f616e01086df19e262af8139b10ead979"}
Jul 15 15:25:53.177392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:53.176952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:53.182933 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:53.182905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx"
Jul 15 15:25:53.209745 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:53.209684    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-25kfx" podStartSLOduration=2.291064897 podStartE2EDuration="4.209663186s" totalImagesPullingTime="1.918598289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:53.208696959 +0000 UTC m=+78.010156774" watchObservedRunningTime="2026-07-15 15:25:53.209663186 +0000 UTC m=+78.011123003"
Jul 15 15:25:53.841376 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:53.841336    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:25:55.186653 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.186612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" event={"ID":"e3ee76aa-7994-48ee-b164-186cdf3ecff1","Type":"ContainerStarted","Data":"c06de1791c0009e7d26109c4d2af70874d84993caf690a0a4e1033549bd36a66"}
Jul 15 15:25:55.187077 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.186715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" event={"ID":"e3ee76aa-7994-48ee-b164-186cdf3ecff1","Type":"ContainerStarted","Data":"85618310e5aceed32758fabbe3602b52a987f66f3d3e8efe5ce9b0cd8037ecbb"}
Jul 15 15:25:55.187077 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.186768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" event={"ID":"e3ee76aa-7994-48ee-b164-186cdf3ecff1","Type":"ContainerStarted","Data":"ecce08e8cad69beedb1449328377d0c95d9e389e4c086d51742a7d9ae38e6f06"}
Jul 15 15:25:55.189716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.189682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerStarted","Data":"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"}
Jul 15 15:25:55.189924 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.189746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerStarted","Data":"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"}
Jul 15 15:25:55.189924 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.189795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerStarted","Data":"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"}
Jul 15 15:25:55.189924 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.189839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerStarted","Data":"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"}
Jul 15 15:25:55.189924 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:55.189864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerStarted","Data":"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"}
Jul 15 15:25:56.197717 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:56.197677    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerStarted","Data":"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"}
Jul 15 15:25:56.233611 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:56.233560    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.477152319 podStartE2EDuration="10.233544353s" totalImagesPullingTime="6.714073868s" totalInitContainerRuntime="42.318166ms" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:56.232527805 +0000 UTC m=+81.033987619" watchObservedRunningTime="2026-07-15 15:25:56.233544353 +0000 UTC m=+81.035004167"
Jul 15 15:25:57.205238 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:57.205151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" event={"ID":"e3ee76aa-7994-48ee-b164-186cdf3ecff1","Type":"ContainerStarted","Data":"65d6a8943dabea7c5cfdcd3de767c0364d24a698d922bfd79370f2848e8ead52"}
Jul 15 15:25:57.205600 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:57.205254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" event={"ID":"e3ee76aa-7994-48ee-b164-186cdf3ecff1","Type":"ContainerStarted","Data":"9079a4d0ba62212c2bc807b0f22ead8c33facfe27bf2f72eca8bde3aa898d2b0"}
Jul 15 15:25:57.205600 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:57.205283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" event={"ID":"e3ee76aa-7994-48ee-b164-186cdf3ecff1","Type":"ContainerStarted","Data":"4dabcf1f2d3da286a4de75d234043ea3a0cc4ee7e2f39caf0045c249aca5c702"}
Jul 15 15:25:57.205687 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:57.205592    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:25:57.230524 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:25:57.230482    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl" podStartSLOduration=2.445103001 podStartE2EDuration="9.230466993s" totalImagesPullingTime="6.785363992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:25:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:25:57.228893248 +0000 UTC m=+82.030353082" watchObservedRunningTime="2026-07-15 15:25:57.230466993 +0000 UTC m=+82.031926808"
Jul 15 15:26:01.311929 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:01.311887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:26:01.312339 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:01.312074    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:26:01.316715 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:01.316692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:26:02.228220 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:02.228191    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:26:02.279097 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:02.279061    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8547f4674-zklnj"]
Jul 15 15:26:03.215025 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:03.215000    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-864774d8b5-ztztl"
Jul 15 15:26:09.788343 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:09.788307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:26:09.788801 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:09.788368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:26:15.981524 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:15.981458    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56c754b4d5-c8n6l" podUID="71c7a16a-3085-4623-a9f5-4f7ab3dc6925" containerName="console" containerID="cri-o://4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e" gracePeriod=15
Jul 15 15:26:16.761602 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.761565    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:26:16.922697 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.922613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-oauth-config\") pod \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") "
Jul 15 15:26:16.922697 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.922655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-config\") pod \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") "
Jul 15 15:26:16.922697 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.922680    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ttng\" (UniqueName: \"kubernetes.io/projected/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-kube-api-access-6ttng\") pod \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") "
Jul 15 15:26:16.922697 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.922699    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-service-ca\") pod \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") "
Jul 15 15:26:16.923080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.922792    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-oauth-serving-cert\") pod \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") "
Jul 15 15:26:16.923080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.922822    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-trusted-ca-bundle\") pod \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") "
Jul 15 15:26:16.923080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.922846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-serving-cert\") pod \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\" (UID: \"71c7a16a-3085-4623-a9f5-4f7ab3dc6925\") "
Jul 15 15:26:16.923243 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.923154    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-config" (OuterVolumeSpecName: "console-config") pod "71c7a16a-3085-4623-a9f5-4f7ab3dc6925" (UID: "71c7a16a-3085-4623-a9f5-4f7ab3dc6925"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:26:16.923243 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.923169    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-service-ca" (OuterVolumeSpecName: "service-ca") pod "71c7a16a-3085-4623-a9f5-4f7ab3dc6925" (UID: "71c7a16a-3085-4623-a9f5-4f7ab3dc6925"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:26:16.923243 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.923224    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "71c7a16a-3085-4623-a9f5-4f7ab3dc6925" (UID: "71c7a16a-3085-4623-a9f5-4f7ab3dc6925"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:26:16.923386 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.923250    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "71c7a16a-3085-4623-a9f5-4f7ab3dc6925" (UID: "71c7a16a-3085-4623-a9f5-4f7ab3dc6925"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:26:16.925240 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.925210    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "71c7a16a-3085-4623-a9f5-4f7ab3dc6925" (UID: "71c7a16a-3085-4623-a9f5-4f7ab3dc6925"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:26:16.925361 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.925338    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "71c7a16a-3085-4623-a9f5-4f7ab3dc6925" (UID: "71c7a16a-3085-4623-a9f5-4f7ab3dc6925"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:26:16.925420 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:16.925359    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-kube-api-access-6ttng" (OuterVolumeSpecName: "kube-api-access-6ttng") pod "71c7a16a-3085-4623-a9f5-4f7ab3dc6925" (UID: "71c7a16a-3085-4623-a9f5-4f7ab3dc6925"). InnerVolumeSpecName "kube-api-access-6ttng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:26:17.024103 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.024071    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-oauth-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:17.024103 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.024102    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:17.024475 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.024114    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ttng\" (UniqueName: \"kubernetes.io/projected/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-kube-api-access-6ttng\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:17.024475 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.024123    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:17.024475 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.024131    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-oauth-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:17.024475 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.024139    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-trusted-ca-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:17.024475 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.024147    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/71c7a16a-3085-4623-a9f5-4f7ab3dc6925-console-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:17.285868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.285791    2580 generic.go:356] "Generic (PLEG): container finished" podID="71c7a16a-3085-4623-a9f5-4f7ab3dc6925" containerID="4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e" exitCode=0
Jul 15 15:26:17.285868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.285835    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c754b4d5-c8n6l"
Jul 15 15:26:17.285868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.285844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c754b4d5-c8n6l" event={"ID":"71c7a16a-3085-4623-a9f5-4f7ab3dc6925","Type":"ContainerDied","Data":"4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e"}
Jul 15 15:26:17.286094 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.285886    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c754b4d5-c8n6l" event={"ID":"71c7a16a-3085-4623-a9f5-4f7ab3dc6925","Type":"ContainerDied","Data":"95ea227e87c90894a0250ae7aae9459726c46fa7409e8b7d2fbc3cac881f253d"}
Jul 15 15:26:17.286094 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.285905    2580 scope.go:117] "RemoveContainer" containerID="4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e"
Jul 15 15:26:17.294901 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.294882    2580 scope.go:117] "RemoveContainer" containerID="4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e"
Jul 15 15:26:17.295173 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:26:17.295155    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e\": container with ID starting with 4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e not found: ID does not exist" containerID="4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e"
Jul 15 15:26:17.295230 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.295182    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e"} err="failed to get container status \"4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e\": rpc error: code = NotFound desc = could not find container \"4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e\": container with ID starting with 4f3b0eeeef4511248924742b7628a1d8fd655b222b03fdf6867e7255ebd1ed1e not found: ID does not exist"
Jul 15 15:26:17.308302 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.308280    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56c754b4d5-c8n6l"]
Jul 15 15:26:17.312063 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.312037    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56c754b4d5-c8n6l"]
Jul 15 15:26:17.766984 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:17.766954    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71c7a16a-3085-4623-a9f5-4f7ab3dc6925" path="/var/lib/kubelet/pods/71c7a16a-3085-4623-a9f5-4f7ab3dc6925/volumes"
Jul 15 15:26:19.146234 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:19.146203    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l87mx"
Jul 15 15:26:27.299549 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.299501    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8547f4674-zklnj" podUID="064b7790-34f6-41af-8e5f-670dfeaae744" containerName="console" containerID="cri-o://9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd" gracePeriod=15
Jul 15 15:26:27.564979 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.564919    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:26:27.711131 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711101    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-service-ca\") pod \"064b7790-34f6-41af-8e5f-670dfeaae744\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") "
Jul 15 15:26:27.711279 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711140    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-serving-cert\") pod \"064b7790-34f6-41af-8e5f-670dfeaae744\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") "
Jul 15 15:26:27.711279 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711165    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtjw9\" (UniqueName: \"kubernetes.io/projected/064b7790-34f6-41af-8e5f-670dfeaae744-kube-api-access-xtjw9\") pod \"064b7790-34f6-41af-8e5f-670dfeaae744\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") "
Jul 15 15:26:27.711381 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-oauth-config\") pod \"064b7790-34f6-41af-8e5f-670dfeaae744\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") "
Jul 15 15:26:27.711437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711411    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-oauth-serving-cert\") pod \"064b7790-34f6-41af-8e5f-670dfeaae744\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") "
Jul 15 15:26:27.711487 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711442    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-console-config\") pod \"064b7790-34f6-41af-8e5f-670dfeaae744\" (UID: \"064b7790-34f6-41af-8e5f-670dfeaae744\") "
Jul 15 15:26:27.711536 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711506    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-service-ca" (OuterVolumeSpecName: "service-ca") pod "064b7790-34f6-41af-8e5f-670dfeaae744" (UID: "064b7790-34f6-41af-8e5f-670dfeaae744"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:26:27.711720 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711703    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:27.711834 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711788    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "064b7790-34f6-41af-8e5f-670dfeaae744" (UID: "064b7790-34f6-41af-8e5f-670dfeaae744"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:26:27.711962 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.711942    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-console-config" (OuterVolumeSpecName: "console-config") pod "064b7790-34f6-41af-8e5f-670dfeaae744" (UID: "064b7790-34f6-41af-8e5f-670dfeaae744"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:26:27.713441 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.713413    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "064b7790-34f6-41af-8e5f-670dfeaae744" (UID: "064b7790-34f6-41af-8e5f-670dfeaae744"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:26:27.713802 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.713784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/064b7790-34f6-41af-8e5f-670dfeaae744-kube-api-access-xtjw9" (OuterVolumeSpecName: "kube-api-access-xtjw9") pod "064b7790-34f6-41af-8e5f-670dfeaae744" (UID: "064b7790-34f6-41af-8e5f-670dfeaae744"). InnerVolumeSpecName "kube-api-access-xtjw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:26:27.713862 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.713808    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "064b7790-34f6-41af-8e5f-670dfeaae744" (UID: "064b7790-34f6-41af-8e5f-670dfeaae744"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:26:27.812611 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.812588    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:27.812611 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.812610    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtjw9\" (UniqueName: \"kubernetes.io/projected/064b7790-34f6-41af-8e5f-670dfeaae744-kube-api-access-xtjw9\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:27.812739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.812622    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/064b7790-34f6-41af-8e5f-670dfeaae744-console-oauth-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:27.812739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.812635    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-oauth-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:27.812739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:27.812644    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/064b7790-34f6-41af-8e5f-670dfeaae744-console-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:26:28.334570 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.334538    2580 generic.go:356] "Generic (PLEG): container finished" podID="064b7790-34f6-41af-8e5f-670dfeaae744" containerID="9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd" exitCode=0
Jul 15 15:26:28.334993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.334604    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8547f4674-zklnj"
Jul 15 15:26:28.334993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.334615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8547f4674-zklnj" event={"ID":"064b7790-34f6-41af-8e5f-670dfeaae744","Type":"ContainerDied","Data":"9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd"}
Jul 15 15:26:28.334993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.334676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8547f4674-zklnj" event={"ID":"064b7790-34f6-41af-8e5f-670dfeaae744","Type":"ContainerDied","Data":"6225992521d09939af4a35a6a79f4945ac43a95dde8b8883ed3a6eea91fc89bb"}
Jul 15 15:26:28.334993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.334696    2580 scope.go:117] "RemoveContainer" containerID="9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd"
Jul 15 15:26:28.342605 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.342587    2580 scope.go:117] "RemoveContainer" containerID="9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd"
Jul 15 15:26:28.342989 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:26:28.342964    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd\": container with ID starting with 9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd not found: ID does not exist" containerID="9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd"
Jul 15 15:26:28.343095 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.342994    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd"} err="failed to get container status \"9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd\": rpc error: code = NotFound desc = could not find container \"9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd\": container with ID starting with 9bb73a8902a38a42357e8f38094f89560386e432789c1d40ae9b11d8867fbfbd not found: ID does not exist"
Jul 15 15:26:28.353827 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.353808    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8547f4674-zklnj"]
Jul 15 15:26:28.358731 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:28.358704    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8547f4674-zklnj"]
Jul 15 15:26:29.766609 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:29.766574    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="064b7790-34f6-41af-8e5f-670dfeaae744" path="/var/lib/kubelet/pods/064b7790-34f6-41af-8e5f-670dfeaae744/volumes"
Jul 15 15:26:29.793949 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:29.793925    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:26:29.797679 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:26:29.797656    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-774f6f965c-45gpn"
Jul 15 15:27:05.686039 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:05.686010    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:27:05.686552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:05.686342    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="alertmanager" containerID="cri-o://abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5" gracePeriod=120
Jul 15 15:27:05.686552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:05.686357    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-web" containerID="cri-o://07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c" gracePeriod=120
Jul 15 15:27:05.686552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:05.686396    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-metric" containerID="cri-o://d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a" gracePeriod=120
Jul 15 15:27:05.686552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:05.686438    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="prom-label-proxy" containerID="cri-o://26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82" gracePeriod=120
Jul 15 15:27:05.686552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:05.686364    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy" containerID="cri-o://4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1" gracePeriod=120
Jul 15 15:27:05.686552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:05.686357    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="config-reloader" containerID="cri-o://92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540" gracePeriod=120
Jul 15 15:27:06.488860 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.488824    2580 generic.go:356] "Generic (PLEG): container finished" podID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerID="26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82" exitCode=0
Jul 15 15:27:06.488860 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.488849    2580 generic.go:356] "Generic (PLEG): container finished" podID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerID="4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1" exitCode=0
Jul 15 15:27:06.488860 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.488855    2580 generic.go:356] "Generic (PLEG): container finished" podID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerID="92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540" exitCode=0
Jul 15 15:27:06.488860 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.488861    2580 generic.go:356] "Generic (PLEG): container finished" podID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerID="abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5" exitCode=0
Jul 15 15:27:06.489108 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.488895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"}
Jul 15 15:27:06.489108 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.488955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"}
Jul 15 15:27:06.489108 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.488984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"}
Jul 15 15:27:06.489108 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.489007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"}
Jul 15 15:27:06.930665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:06.930643    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.033438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033363    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-tls-assets\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033417    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-trusted-ca-bundle\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033444    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033480    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033507    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-main-db\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033619    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-web-config\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033671    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033698    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-out\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033728    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw2p9\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-kube-api-access-jw2p9\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033779    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-cluster-tls-config\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033788    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:27:07.033868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.033868 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-volume\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.034224 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033892    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-metrics-client-ca\") pod \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\" (UID: \"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e\") "
Jul 15 15:27:07.034224 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.033902    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:27:07.034224 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.034159    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.034224 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.034179    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-alertmanager-main-db\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.035309 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.035268    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:27:07.037142 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.037112    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:07.037744 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.037713    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:07.037864 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.037817    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:07.038062 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.038041    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:07.038724 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.038694    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:27:07.038901 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.038873    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-kube-api-access-jw2p9" (OuterVolumeSpecName: "kube-api-access-jw2p9") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "kube-api-access-jw2p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:27:07.039051 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.039027    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-out" (OuterVolumeSpecName: "config-out") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:27:07.039284 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.039259    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-volume" (OuterVolumeSpecName: "config-volume") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:07.041687 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.041663    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:07.049650 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.049628    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-web-config" (OuterVolumeSpecName: "web-config") pod "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" (UID: "ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:07.134727 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134699    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-metrics-client-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134727 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134725    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-tls-assets\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134735    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-main-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134745    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134772    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-web-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134780    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134789    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-out\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134798    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw2p9\" (UniqueName: \"kubernetes.io/projected/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-kube-api-access-jw2p9\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134805    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-cluster-tls-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134813    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.134892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.134822    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e-config-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:07.495136 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.495105    2580 generic.go:356] "Generic (PLEG): container finished" podID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerID="d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a" exitCode=0
Jul 15 15:27:07.495136 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.495128    2580 generic.go:356] "Generic (PLEG): container finished" podID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerID="07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c" exitCode=0
Jul 15 15:27:07.495391 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.495171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"}
Jul 15 15:27:07.495391 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.495193    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.495391 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.495262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"}
Jul 15 15:27:07.495391 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.495292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e","Type":"ContainerDied","Data":"11bba41cce4869c78f3a793dfc4235b249776666b46c96ec65e42705a7afa3c2"}
Jul 15 15:27:07.495391 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.495332    2580 scope.go:117] "RemoveContainer" containerID="26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"
Jul 15 15:27:07.502897 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.502793    2580 scope.go:117] "RemoveContainer" containerID="d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"
Jul 15 15:27:07.509426 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.509408    2580 scope.go:117] "RemoveContainer" containerID="4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"
Jul 15 15:27:07.515839 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.515825    2580 scope.go:117] "RemoveContainer" containerID="07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"
Jul 15 15:27:07.521943 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.521915    2580 scope.go:117] "RemoveContainer" containerID="92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"
Jul 15 15:27:07.528744 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.528725    2580 scope.go:117] "RemoveContainer" containerID="abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"
Jul 15 15:27:07.535727 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.535710    2580 scope.go:117] "RemoveContainer" containerID="dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1"
Jul 15 15:27:07.537492 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.537467    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:27:07.542273 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.542257    2580 scope.go:117] "RemoveContainer" containerID="26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"
Jul 15 15:27:07.542516 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:07.542497    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82\": container with ID starting with 26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82 not found: ID does not exist" containerID="26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"
Jul 15 15:27:07.542582 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.542527    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"} err="failed to get container status \"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82\": rpc error: code = NotFound desc = could not find container \"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82\": container with ID starting with 26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82 not found: ID does not exist"
Jul 15 15:27:07.542582 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.542551    2580 scope.go:117] "RemoveContainer" containerID="d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"
Jul 15 15:27:07.542787 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:07.542771    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a\": container with ID starting with d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a not found: ID does not exist" containerID="d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"
Jul 15 15:27:07.542838 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.542792    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"} err="failed to get container status \"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a\": rpc error: code = NotFound desc = could not find container \"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a\": container with ID starting with d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a not found: ID does not exist"
Jul 15 15:27:07.542838 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.542805    2580 scope.go:117] "RemoveContainer" containerID="4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"
Jul 15 15:27:07.543100 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:07.543082    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1\": container with ID starting with 4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1 not found: ID does not exist" containerID="4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"
Jul 15 15:27:07.543151 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543103    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"} err="failed to get container status \"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1\": rpc error: code = NotFound desc = could not find container \"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1\": container with ID starting with 4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1 not found: ID does not exist"
Jul 15 15:27:07.543151 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543118    2580 scope.go:117] "RemoveContainer" containerID="07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"
Jul 15 15:27:07.543386 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:07.543360    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c\": container with ID starting with 07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c not found: ID does not exist" containerID="07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"
Jul 15 15:27:07.543426 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543381    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"} err="failed to get container status \"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c\": rpc error: code = NotFound desc = could not find container \"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c\": container with ID starting with 07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c not found: ID does not exist"
Jul 15 15:27:07.543426 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543399    2580 scope.go:117] "RemoveContainer" containerID="92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"
Jul 15 15:27:07.543632 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:07.543614    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540\": container with ID starting with 92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540 not found: ID does not exist" containerID="92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"
Jul 15 15:27:07.543682 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543637    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"} err="failed to get container status \"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540\": rpc error: code = NotFound desc = could not find container \"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540\": container with ID starting with 92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540 not found: ID does not exist"
Jul 15 15:27:07.543682 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543656    2580 scope.go:117] "RemoveContainer" containerID="abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"
Jul 15 15:27:07.543914 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:07.543898    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5\": container with ID starting with abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5 not found: ID does not exist" containerID="abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"
Jul 15 15:27:07.543968 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543918    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"} err="failed to get container status \"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5\": rpc error: code = NotFound desc = could not find container \"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5\": container with ID starting with abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5 not found: ID does not exist"
Jul 15 15:27:07.543968 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.543932    2580 scope.go:117] "RemoveContainer" containerID="dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1"
Jul 15 15:27:07.544166 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:07.544150    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1\": container with ID starting with dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1 not found: ID does not exist" containerID="dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1"
Jul 15 15:27:07.544218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544170    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1"} err="failed to get container status \"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1\": rpc error: code = NotFound desc = could not find container \"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1\": container with ID starting with dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1 not found: ID does not exist"
Jul 15 15:27:07.544218 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544182    2580 scope.go:117] "RemoveContainer" containerID="26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"
Jul 15 15:27:07.544444 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544392    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82"} err="failed to get container status \"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82\": rpc error: code = NotFound desc = could not find container \"26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82\": container with ID starting with 26795cc39613eeee8ffe971bdafc35e832c3aa6c0dcb3b28ecec350a2bfd1f82 not found: ID does not exist"
Jul 15 15:27:07.544444 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544419    2580 scope.go:117] "RemoveContainer" containerID="d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"
Jul 15 15:27:07.544663 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544638    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a"} err="failed to get container status \"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a\": rpc error: code = NotFound desc = could not find container \"d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a\": container with ID starting with d27de00e41ed9fe846a24661494cae654f8ab73eca61438c9638ad51a7b5064a not found: ID does not exist"
Jul 15 15:27:07.544663 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544663    2580 scope.go:117] "RemoveContainer" containerID="4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"
Jul 15 15:27:07.544941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544921    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1"} err="failed to get container status \"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1\": rpc error: code = NotFound desc = could not find container \"4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1\": container with ID starting with 4d63d4e8bf9d36ff35b6231978ef90dcaa2839d1f9d4eed8f34f2b5e540f8ec1 not found: ID does not exist"
Jul 15 15:27:07.544941 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.544939    2580 scope.go:117] "RemoveContainer" containerID="07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"
Jul 15 15:27:07.545173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.545155    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c"} err="failed to get container status \"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c\": rpc error: code = NotFound desc = could not find container \"07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c\": container with ID starting with 07a3ad6695ef54de025eb0a44ff37e30551a391f2ee0d864a968d522e8e7af3c not found: ID does not exist"
Jul 15 15:27:07.545173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.545173    2580 scope.go:117] "RemoveContainer" containerID="92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"
Jul 15 15:27:07.545405 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.545387    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540"} err="failed to get container status \"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540\": rpc error: code = NotFound desc = could not find container \"92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540\": container with ID starting with 92f35d4b4455f2e6add3fedf33611426ee1b08ce706fa3d2dede107b21d8c540 not found: ID does not exist"
Jul 15 15:27:07.545460 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.545406    2580 scope.go:117] "RemoveContainer" containerID="abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"
Jul 15 15:27:07.545654 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.545637    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5"} err="failed to get container status \"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5\": rpc error: code = NotFound desc = could not find container \"abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5\": container with ID starting with abce7c457487c5e724901ed8cb71e6435d87591c6c9aac8724cb83fddca8d4c5 not found: ID does not exist"
Jul 15 15:27:07.545699 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.545654    2580 scope.go:117] "RemoveContainer" containerID="dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1"
Jul 15 15:27:07.545929 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.545911    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1"} err="failed to get container status \"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1\": rpc error: code = NotFound desc = could not find container \"dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1\": container with ID starting with dd10b9b316b05ee454c95d8ef5f8d71f68fda9ac59065e07af3ae62c3b24e0e1 not found: ID does not exist"
Jul 15 15:27:07.547976 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.547951    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:27:07.608704 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.608679    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:27:07.609030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.608993    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="config-reloader"
Jul 15 15:27:07.609030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609008    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="config-reloader"
Jul 15 15:27:07.609030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609016    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-web"
Jul 15 15:27:07.609030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609021    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-web"
Jul 15 15:27:07.609030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609027    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="prom-label-proxy"
Jul 15 15:27:07.609030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609032    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="prom-label-proxy"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609041    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71c7a16a-3085-4623-a9f5-4f7ab3dc6925" containerName="console"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609046    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71c7a16a-3085-4623-a9f5-4f7ab3dc6925" containerName="console"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609053    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="064b7790-34f6-41af-8e5f-670dfeaae744" containerName="console"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609058    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="064b7790-34f6-41af-8e5f-670dfeaae744" containerName="console"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609065    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="alertmanager"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609070    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="alertmanager"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609079    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="init-config-reloader"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609084    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="init-config-reloader"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609090    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-metric"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609095    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-metric"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609101    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609107    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609149    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="config-reloader"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609156    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="alertmanager"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609164    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="064b7790-34f6-41af-8e5f-670dfeaae744" containerName="console"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609170    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609176    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="prom-label-proxy"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609181    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-metric"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609187    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" containerName="kube-rbac-proxy-web"
Jul 15 15:27:07.609248 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.609196    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="71c7a16a-3085-4623-a9f5-4f7ab3dc6925" containerName="console"
Jul 15 15:27:07.613145 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.613131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.624563 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.624529    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 15:27:07.624825 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.624811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 15:27:07.624902 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.624825    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 15:27:07.624902 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.624828    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 15:27:07.624978 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.624949    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 15:27:07.625981 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.625967    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 15:27:07.626180 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.626164    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 15:27:07.626180 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.626174    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 15:27:07.626317 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.626298    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rcwpr\""
Jul 15 15:27:07.640837 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.640819    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 15:27:07.656532 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.656509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:27:07.738394 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc19423b-4abd-4cb3-a8b9-126734fdadaa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738394 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738562 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc19423b-4abd-4cb3-a8b9-126734fdadaa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738562 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738562 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738562 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc19423b-4abd-4cb3-a8b9-126734fdadaa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738562 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc19423b-4abd-4cb3-a8b9-126734fdadaa-config-out\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc19423b-4abd-4cb3-a8b9-126734fdadaa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-web-config\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.738893 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.738717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk6w2\" (UniqueName: \"kubernetes.io/projected/cc19423b-4abd-4cb3-a8b9-126734fdadaa-kube-api-access-dk6w2\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.769630 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.769602    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e" path="/var/lib/kubelet/pods/ceaac3d0-c0dc-4213-a37b-e53d1fd44d3e/volumes"
Jul 15 15:27:07.839716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.839683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-web-config\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.839894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.839725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.839894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.839771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.839894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.839798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk6w2\" (UniqueName: \"kubernetes.io/projected/cc19423b-4abd-4cb3-a8b9-126734fdadaa-kube-api-access-dk6w2\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.839894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.839852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc19423b-4abd-4cb3-a8b9-126734fdadaa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.839894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.839881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840464 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.839907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc19423b-4abd-4cb3-a8b9-126734fdadaa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840464 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840464 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840464 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc19423b-4abd-4cb3-a8b9-126734fdadaa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840464 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc19423b-4abd-4cb3-a8b9-126734fdadaa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840713 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc19423b-4abd-4cb3-a8b9-126734fdadaa-config-out\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840713 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840713 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc19423b-4abd-4cb3-a8b9-126734fdadaa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.840713 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.840664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc19423b-4abd-4cb3-a8b9-126734fdadaa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.841853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.841502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc19423b-4abd-4cb3-a8b9-126734fdadaa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.842994 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.842974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.843133 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.843096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.843567 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.843423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-web-config\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.843567 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.843512    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.843567 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.843519    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.843567 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.843515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc19423b-4abd-4cb3-a8b9-126734fdadaa-config-out\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.843735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.843629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc19423b-4abd-4cb3-a8b9-126734fdadaa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.844394 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.844376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.844929 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.844915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc19423b-4abd-4cb3-a8b9-126734fdadaa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.850219 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.850199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk6w2\" (UniqueName: \"kubernetes.io/projected/cc19423b-4abd-4cb3-a8b9-126734fdadaa-kube-api-access-dk6w2\") pod \"alertmanager-main-0\" (UID: \"cc19423b-4abd-4cb3-a8b9-126734fdadaa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:07.921262 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:07.921237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 15:27:08.078681 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:08.078581    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 15:27:08.082237 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:27:08.082209    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc19423b_4abd_4cb3_a8b9_126734fdadaa.slice/crio-8e6e3d9785fd133fcc0340def86075a79b189ee17804a576a70851f29642806e WatchSource:0}: Error finding container 8e6e3d9785fd133fcc0340def86075a79b189ee17804a576a70851f29642806e: Status 404 returned error can't find the container with id 8e6e3d9785fd133fcc0340def86075a79b189ee17804a576a70851f29642806e
Jul 15 15:27:08.504931 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:08.504842    2580 generic.go:356] "Generic (PLEG): container finished" podID="cc19423b-4abd-4cb3-a8b9-126734fdadaa" containerID="8252ee76d0ff9c9c138d1ee9af449bb9fc80f63f5b5c4ed6037f758ffe06b258" exitCode=0
Jul 15 15:27:08.505062 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:08.504920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerDied","Data":"8252ee76d0ff9c9c138d1ee9af449bb9fc80f63f5b5c4ed6037f758ffe06b258"}
Jul 15 15:27:08.505062 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:08.505007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerStarted","Data":"8e6e3d9785fd133fcc0340def86075a79b189ee17804a576a70851f29642806e"}
Jul 15 15:27:09.512086 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.512048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerStarted","Data":"d8bd58fef7dca70ed85b486714d955e5900f7f0e5d97454779d1aa6dffa1709f"}
Jul 15 15:27:09.512497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.512131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerStarted","Data":"49ca0a3f26b77ad83b7f9cee97362e2aaa8a0367d5fd2d6e71f1048773764e27"}
Jul 15 15:27:09.512497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.512176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerStarted","Data":"00461fd3a18b6c0ef74a5f6dfa77fc75df81c85697a653dbf53d5a425c30f207"}
Jul 15 15:27:09.512497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.512210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerStarted","Data":"2cdb53be8a9d14d1201042818c57ca69ecffd39e53cca75a01111c953cb8842d"}
Jul 15 15:27:09.512497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.512247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerStarted","Data":"9cde5c114c3b39640d287fb7e4767e437b8341a03a61d6f6657d0b6bd8f456e9"}
Jul 15 15:27:09.512497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.512289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc19423b-4abd-4cb3-a8b9-126734fdadaa","Type":"ContainerStarted","Data":"ba4458e85af054e6e4208479d5c8368563a857e3c7c7383696d992bb290a69b5"}
Jul 15 15:27:09.561454 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.561404    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.546291121 podStartE2EDuration="2.561384979s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.093858ms" isStatefulPod=false podCreationTimestamp="2026-07-15 15:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:27:09.560313951 +0000 UTC m=+154.361773768" watchObservedRunningTime="2026-07-15 15:27:09.561384979 +0000 UTC m=+154.362844793"
Jul 15 15:27:09.816224 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.816146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"]
Jul 15 15:27:09.818723 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.818702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.823394 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.823349    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w9bf5\""
Jul 15 15:27:09.823394 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.823367    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 15 15:27:09.823561 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.823406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 15 15:27:09.823561 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.823371    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 15 15:27:09.823664 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.823654    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 15 15:27:09.823715 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.823673    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 15 15:27:09.832287 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.832266    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 15 15:27:09.854657 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.854637    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"]
Jul 15 15:27:09.959513 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-telemeter-client-tls\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.959513 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-federate-client-tls\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.959710 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-metrics-client-ca\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.959710 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.959710 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-serving-certs-ca-bundle\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.959710 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-secret-telemeter-client\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.959866 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvq46\" (UniqueName: \"kubernetes.io/projected/e677916f-866a-4fc4-9f68-06c938d16d30-kube-api-access-jvq46\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:09.959866 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:09.959793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060421 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-secret-telemeter-client\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvq46\" (UniqueName: \"kubernetes.io/projected/e677916f-866a-4fc4-9f68-06c938d16d30-kube-api-access-jvq46\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-telemeter-client-tls\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-federate-client-tls\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-metrics-client-ca\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.060632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.060603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-serving-certs-ca-bundle\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.061384 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.061355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-metrics-client-ca\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.061488 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.061402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-serving-certs-ca-bundle\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.061806 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.061778    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e677916f-866a-4fc4-9f68-06c938d16d30-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.063091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.063066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.063256 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.063239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-federate-client-tls\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.063335 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.063320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-telemeter-client-tls\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.063493 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.063473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e677916f-866a-4fc4-9f68-06c938d16d30-secret-telemeter-client\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.074952 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.074931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvq46\" (UniqueName: \"kubernetes.io/projected/e677916f-866a-4fc4-9f68-06c938d16d30-kube-api-access-jvq46\") pod \"telemeter-client-6cb9b7db46-lmzq7\" (UID: \"e677916f-866a-4fc4-9f68-06c938d16d30\") " pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.128047 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.128021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"
Jul 15 15:27:10.312814 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.312711    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7"]
Jul 15 15:27:10.316017 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:27:10.315970    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode677916f_866a_4fc4_9f68_06c938d16d30.slice/crio-b76275a684d0669b2d03e7552ff7baefc51dc6fcbf58e6a9e4bdcc7029783cde WatchSource:0}: Error finding container b76275a684d0669b2d03e7552ff7baefc51dc6fcbf58e6a9e4bdcc7029783cde: Status 404 returned error can't find the container with id b76275a684d0669b2d03e7552ff7baefc51dc6fcbf58e6a9e4bdcc7029783cde
Jul 15 15:27:10.523063 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:10.522962    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7" event={"ID":"e677916f-866a-4fc4-9f68-06c938d16d30","Type":"ContainerStarted","Data":"b76275a684d0669b2d03e7552ff7baefc51dc6fcbf58e6a9e4bdcc7029783cde"}
Jul 15 15:27:14.542679 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:14.542648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7" event={"ID":"e677916f-866a-4fc4-9f68-06c938d16d30","Type":"ContainerStarted","Data":"e0d25a66b1cca173508016e8cd74e66e6b91bf5980361283a7cda29a57c68acd"}
Jul 15 15:27:14.543066 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:14.542706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7" event={"ID":"e677916f-866a-4fc4-9f68-06c938d16d30","Type":"ContainerStarted","Data":"cb551aacc4c6bfa41a71b884a1c01cb02e3ad4c724824dae45a60ce38f085b00"}
Jul 15 15:27:14.543066 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:14.542732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7" event={"ID":"e677916f-866a-4fc4-9f68-06c938d16d30","Type":"ContainerStarted","Data":"7fb2a168a21e7f1b92216b23d41bb6a8dc6db9447880da6d5579939147b49fe2"}
Jul 15 15:27:14.584295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:14.584249    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cb9b7db46-lmzq7" podStartSLOduration=2.387657635 podStartE2EDuration="5.58423415s" totalImagesPullingTime="3.196576515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:27:14.58056805 +0000 UTC m=+159.382027864" watchObservedRunningTime="2026-07-15 15:27:14.58423415 +0000 UTC m=+159.385693964"
Jul 15 15:27:15.330768 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.330711    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b5b79d4b-ljcpz"]
Jul 15 15:27:15.333016 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.332996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.355806 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.355777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b5b79d4b-ljcpz"]
Jul 15 15:27:15.408897 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.408869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-oauth-serving-cert\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.408997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.408905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-trusted-ca-bundle\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.408997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.408932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-service-ca\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.409073 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.409007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-serving-cert\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.409073 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.409036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-oauth-config\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.409140 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.409073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22kw2\" (UniqueName: \"kubernetes.io/projected/13ee8539-3140-42dc-b190-f294855304ca-kube-api-access-22kw2\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.409140 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.409119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-console-config\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.510441 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.510407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-oauth-serving-cert\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.510441 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.510448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-trusted-ca-bundle\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.510559 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.510470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-service-ca\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.510559 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.510499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-serving-cert\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.510559 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.510519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-oauth-config\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.510559 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.510550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22kw2\" (UniqueName: \"kubernetes.io/projected/13ee8539-3140-42dc-b190-f294855304ca-kube-api-access-22kw2\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.510717 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.510573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-console-config\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.511270 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.511242    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-console-config\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.511360 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.511253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-oauth-serving-cert\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.511360 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.511311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-trusted-ca-bundle\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.511360 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.511343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-service-ca\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.512923 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.512902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-oauth-config\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.513175 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.513159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-serving-cert\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.520125 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.520100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22kw2\" (UniqueName: \"kubernetes.io/projected/13ee8539-3140-42dc-b190-f294855304ca-kube-api-access-22kw2\") pod \"console-77b5b79d4b-ljcpz\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") " pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.643667 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.643602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:15.780240 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:15.780209    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b5b79d4b-ljcpz"]
Jul 15 15:27:15.782603 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:27:15.782581    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ee8539_3140_42dc_b190_f294855304ca.slice/crio-e239b1dc88334a56503e81c34c1b01e3442f5b75158b43a4ebe2c9c362d0d998 WatchSource:0}: Error finding container e239b1dc88334a56503e81c34c1b01e3442f5b75158b43a4ebe2c9c362d0d998: Status 404 returned error can't find the container with id e239b1dc88334a56503e81c34c1b01e3442f5b75158b43a4ebe2c9c362d0d998
Jul 15 15:27:16.552336 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:16.552301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5b79d4b-ljcpz" event={"ID":"13ee8539-3140-42dc-b190-f294855304ca","Type":"ContainerStarted","Data":"784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0"}
Jul 15 15:27:16.552515 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:16.552352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5b79d4b-ljcpz" event={"ID":"13ee8539-3140-42dc-b190-f294855304ca","Type":"ContainerStarted","Data":"e239b1dc88334a56503e81c34c1b01e3442f5b75158b43a4ebe2c9c362d0d998"}
Jul 15 15:27:16.580567 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:16.580517    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b5b79d4b-ljcpz" podStartSLOduration=1.5805015519999999 podStartE2EDuration="1.580501552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:27:16.579352682 +0000 UTC m=+161.380812499" watchObservedRunningTime="2026-07-15 15:27:16.580501552 +0000 UTC m=+161.381961365"
Jul 15 15:27:25.644096 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:25.644065    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:25.644475 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:25.644157    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:25.649379 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:25.649355    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:26.593739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:26.593711    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:27:26.654359 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:26.654324    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ffc5689d4-9swcx"]
Jul 15 15:27:51.675067 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:51.675009    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5ffc5689d4-9swcx" podUID="a763b861-2044-4f24-9ed0-801cc90ba91d" containerName="console" containerID="cri-o://0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90" gracePeriod=15
Jul 15 15:27:51.915888 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:51.915861    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:27:52.019775 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.019671    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpj4b\" (UniqueName: \"kubernetes.io/projected/a763b861-2044-4f24-9ed0-801cc90ba91d-kube-api-access-xpj4b\") pod \"a763b861-2044-4f24-9ed0-801cc90ba91d\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") "
Jul 15 15:27:52.019775 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.019709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-oauth-config\") pod \"a763b861-2044-4f24-9ed0-801cc90ba91d\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") "
Jul 15 15:27:52.019775 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.019739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-trusted-ca-bundle\") pod \"a763b861-2044-4f24-9ed0-801cc90ba91d\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") "
Jul 15 15:27:52.020049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.019779    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-console-config\") pod \"a763b861-2044-4f24-9ed0-801cc90ba91d\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") "
Jul 15 15:27:52.020049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.019829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-serving-cert\") pod \"a763b861-2044-4f24-9ed0-801cc90ba91d\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") "
Jul 15 15:27:52.020049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.019854    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-oauth-serving-cert\") pod \"a763b861-2044-4f24-9ed0-801cc90ba91d\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") "
Jul 15 15:27:52.020049 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.019905    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-service-ca\") pod \"a763b861-2044-4f24-9ed0-801cc90ba91d\" (UID: \"a763b861-2044-4f24-9ed0-801cc90ba91d\") "
Jul 15 15:27:52.020290 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.020256    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a763b861-2044-4f24-9ed0-801cc90ba91d" (UID: "a763b861-2044-4f24-9ed0-801cc90ba91d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:27:52.020290 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.020264    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-console-config" (OuterVolumeSpecName: "console-config") pod "a763b861-2044-4f24-9ed0-801cc90ba91d" (UID: "a763b861-2044-4f24-9ed0-801cc90ba91d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:27:52.020290 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.020284    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a763b861-2044-4f24-9ed0-801cc90ba91d" (UID: "a763b861-2044-4f24-9ed0-801cc90ba91d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:27:52.020578 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.020548    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-service-ca" (OuterVolumeSpecName: "service-ca") pod "a763b861-2044-4f24-9ed0-801cc90ba91d" (UID: "a763b861-2044-4f24-9ed0-801cc90ba91d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:27:52.022023 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.022000    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a763b861-2044-4f24-9ed0-801cc90ba91d-kube-api-access-xpj4b" (OuterVolumeSpecName: "kube-api-access-xpj4b") pod "a763b861-2044-4f24-9ed0-801cc90ba91d" (UID: "a763b861-2044-4f24-9ed0-801cc90ba91d"). InnerVolumeSpecName "kube-api-access-xpj4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:27:52.022023 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.022013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a763b861-2044-4f24-9ed0-801cc90ba91d" (UID: "a763b861-2044-4f24-9ed0-801cc90ba91d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:52.022188 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.022038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a763b861-2044-4f24-9ed0-801cc90ba91d" (UID: "a763b861-2044-4f24-9ed0-801cc90ba91d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:27:52.121285 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.121260    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-trusted-ca-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:52.121285 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.121281    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-console-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:52.121435 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.121290    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:52.121435 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.121299    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-oauth-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:52.121435 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.121308    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a763b861-2044-4f24-9ed0-801cc90ba91d-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:52.121435 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.121317    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpj4b\" (UniqueName: \"kubernetes.io/projected/a763b861-2044-4f24-9ed0-801cc90ba91d-kube-api-access-xpj4b\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:52.121435 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.121325    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a763b861-2044-4f24-9ed0-801cc90ba91d-console-oauth-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:27:52.690321 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.690284    2580 generic.go:356] "Generic (PLEG): container finished" podID="a763b861-2044-4f24-9ed0-801cc90ba91d" containerID="0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90" exitCode=0
Jul 15 15:27:52.690701 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.690334    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ffc5689d4-9swcx"
Jul 15 15:27:52.690701 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.690351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ffc5689d4-9swcx" event={"ID":"a763b861-2044-4f24-9ed0-801cc90ba91d","Type":"ContainerDied","Data":"0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90"}
Jul 15 15:27:52.690701 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.690413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ffc5689d4-9swcx" event={"ID":"a763b861-2044-4f24-9ed0-801cc90ba91d","Type":"ContainerDied","Data":"8c5910bc28b60f4361cacae15b0a167ea5d9c4fcd38103bb2f6b6267c4201936"}
Jul 15 15:27:52.690701 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.690434    2580 scope.go:117] "RemoveContainer" containerID="0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90"
Jul 15 15:27:52.699342 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.699318    2580 scope.go:117] "RemoveContainer" containerID="0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90"
Jul 15 15:27:52.699612 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:27:52.699592    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90\": container with ID starting with 0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90 not found: ID does not exist" containerID="0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90"
Jul 15 15:27:52.699673 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.699625    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90"} err="failed to get container status \"0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90\": rpc error: code = NotFound desc = could not find container \"0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90\": container with ID starting with 0e952da552498cc9c3b8ac1b4fd02debc8e47f74de8ce6b140c15fbd0e01bc90 not found: ID does not exist"
Jul 15 15:27:52.715380 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.715359    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ffc5689d4-9swcx"]
Jul 15 15:27:52.728546 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:52.728522    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5ffc5689d4-9swcx"]
Jul 15 15:27:53.766242 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:27:53.766202    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a763b861-2044-4f24-9ed0-801cc90ba91d" path="/var/lib/kubelet/pods/a763b861-2044-4f24-9ed0-801cc90ba91d/volumes"
Jul 15 15:28:23.051642 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.051595    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8bcdc5ccc-dk78x"]
Jul 15 15:28:23.052453 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.052428    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a763b861-2044-4f24-9ed0-801cc90ba91d" containerName="console"
Jul 15 15:28:23.052542 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.052455    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a763b861-2044-4f24-9ed0-801cc90ba91d" containerName="console"
Jul 15 15:28:23.052635 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.052622    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a763b861-2044-4f24-9ed0-801cc90ba91d" containerName="console"
Jul 15 15:28:23.056542 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.056520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.106143 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.106108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bcdc5ccc-dk78x"]
Jul 15 15:28:23.163898 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.163867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-trusted-ca-bundle\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.164033 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.163905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-serving-cert\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.164033 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.163922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-oauth-config\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.164033 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.164001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-service-ca\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.164033 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.164031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-oauth-serving-cert\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.164184 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.164067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptth8\" (UniqueName: \"kubernetes.io/projected/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-kube-api-access-ptth8\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.164184 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.164084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-config\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.265195 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.265163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptth8\" (UniqueName: \"kubernetes.io/projected/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-kube-api-access-ptth8\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.265414 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.265203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-config\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.265414 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.265247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-trusted-ca-bundle\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.265414 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.265284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-serving-cert\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.265414 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.265308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-oauth-config\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.265414 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.265373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-service-ca\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.265414 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.265399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-oauth-serving-cert\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.266129 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.266096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-config\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.266244 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.266098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-service-ca\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.266244 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.266178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-trusted-ca-bundle\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.266244 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.266182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-oauth-serving-cert\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.267812 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.267785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-serving-cert\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.267896 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.267821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-oauth-config\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.291959 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.291936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptth8\" (UniqueName: \"kubernetes.io/projected/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-kube-api-access-ptth8\") pod \"console-8bcdc5ccc-dk78x\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") " pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.365483 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.365457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:23.509767 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.509721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bcdc5ccc-dk78x"]
Jul 15 15:28:23.512599 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:28:23.512570    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ca15b2_7aec_42a4_a2cb_a7c28fb3cb5b.slice/crio-399f47381fd440fe75a5b180343b0403345fba6f43efa8e7df360fa42314ca6c WatchSource:0}: Error finding container 399f47381fd440fe75a5b180343b0403345fba6f43efa8e7df360fa42314ca6c: Status 404 returned error can't find the container with id 399f47381fd440fe75a5b180343b0403345fba6f43efa8e7df360fa42314ca6c
Jul 15 15:28:23.816338 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.816256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcdc5ccc-dk78x" event={"ID":"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b","Type":"ContainerStarted","Data":"9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3"}
Jul 15 15:28:23.816338 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:23.816300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcdc5ccc-dk78x" event={"ID":"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b","Type":"ContainerStarted","Data":"399f47381fd440fe75a5b180343b0403345fba6f43efa8e7df360fa42314ca6c"}
Jul 15 15:28:33.366378 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:33.366346    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:33.366790 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:33.366401    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:33.371062 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:33.371039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:33.412879 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:33.412836    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8bcdc5ccc-dk78x" podStartSLOduration=10.412824004 podStartE2EDuration="10.412824004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:28:23.854948128 +0000 UTC m=+228.656407962" watchObservedRunningTime="2026-07-15 15:28:33.412824004 +0000 UTC m=+238.214283820"
Jul 15 15:28:33.858171 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:33.858145    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:28:33.943623 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:33.943598    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b5b79d4b-ljcpz"]
Jul 15 15:28:58.963712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:58.963605    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b5b79d4b-ljcpz" podUID="13ee8539-3140-42dc-b190-f294855304ca" containerName="console" containerID="cri-o://784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0" gracePeriod=15
Jul 15 15:28:59.204479 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.204455    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:28:59.364340 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-oauth-config\") pod \"13ee8539-3140-42dc-b190-f294855304ca\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") "
Jul 15 15:28:59.364485 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364368    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-trusted-ca-bundle\") pod \"13ee8539-3140-42dc-b190-f294855304ca\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") "
Jul 15 15:28:59.364485 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364403    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-service-ca\") pod \"13ee8539-3140-42dc-b190-f294855304ca\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") "
Jul 15 15:28:59.364485 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364422    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22kw2\" (UniqueName: \"kubernetes.io/projected/13ee8539-3140-42dc-b190-f294855304ca-kube-api-access-22kw2\") pod \"13ee8539-3140-42dc-b190-f294855304ca\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") "
Jul 15 15:28:59.364485 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-serving-cert\") pod \"13ee8539-3140-42dc-b190-f294855304ca\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") "
Jul 15 15:28:59.364678 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-oauth-serving-cert\") pod \"13ee8539-3140-42dc-b190-f294855304ca\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") "
Jul 15 15:28:59.364678 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364558    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-console-config\") pod \"13ee8539-3140-42dc-b190-f294855304ca\" (UID: \"13ee8539-3140-42dc-b190-f294855304ca\") "
Jul 15 15:28:59.364930 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364876    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-service-ca" (OuterVolumeSpecName: "service-ca") pod "13ee8539-3140-42dc-b190-f294855304ca" (UID: "13ee8539-3140-42dc-b190-f294855304ca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:28:59.365052 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.364988    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "13ee8539-3140-42dc-b190-f294855304ca" (UID: "13ee8539-3140-42dc-b190-f294855304ca"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:28:59.365052 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.365023    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "13ee8539-3140-42dc-b190-f294855304ca" (UID: "13ee8539-3140-42dc-b190-f294855304ca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:28:59.365173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.365098    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-console-config" (OuterVolumeSpecName: "console-config") pod "13ee8539-3140-42dc-b190-f294855304ca" (UID: "13ee8539-3140-42dc-b190-f294855304ca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:28:59.366549 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.366523    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ee8539-3140-42dc-b190-f294855304ca-kube-api-access-22kw2" (OuterVolumeSpecName: "kube-api-access-22kw2") pod "13ee8539-3140-42dc-b190-f294855304ca" (UID: "13ee8539-3140-42dc-b190-f294855304ca"). InnerVolumeSpecName "kube-api-access-22kw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:28:59.366649 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.366561    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "13ee8539-3140-42dc-b190-f294855304ca" (UID: "13ee8539-3140-42dc-b190-f294855304ca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:28:59.366649 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.366586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "13ee8539-3140-42dc-b190-f294855304ca" (UID: "13ee8539-3140-42dc-b190-f294855304ca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:28:59.465838 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.465812    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:28:59.465838 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.465836    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22kw2\" (UniqueName: \"kubernetes.io/projected/13ee8539-3140-42dc-b190-f294855304ca-kube-api-access-22kw2\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:28:59.465976 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.465847    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:28:59.465976 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.465857    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-oauth-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:28:59.465976 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.465866    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-console-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:28:59.465976 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.465875    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13ee8539-3140-42dc-b190-f294855304ca-console-oauth-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:28:59.465976 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.465883    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ee8539-3140-42dc-b190-f294855304ca-trusted-ca-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:28:59.954637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.954602    2580 generic.go:356] "Generic (PLEG): container finished" podID="13ee8539-3140-42dc-b190-f294855304ca" containerID="784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0" exitCode=0
Jul 15 15:28:59.954877 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.954643    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b5b79d4b-ljcpz"
Jul 15 15:28:59.954877 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.954641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5b79d4b-ljcpz" event={"ID":"13ee8539-3140-42dc-b190-f294855304ca","Type":"ContainerDied","Data":"784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0"}
Jul 15 15:28:59.954877 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.954786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b5b79d4b-ljcpz" event={"ID":"13ee8539-3140-42dc-b190-f294855304ca","Type":"ContainerDied","Data":"e239b1dc88334a56503e81c34c1b01e3442f5b75158b43a4ebe2c9c362d0d998"}
Jul 15 15:28:59.954877 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.954815    2580 scope.go:117] "RemoveContainer" containerID="784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0"
Jul 15 15:28:59.962688 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.962673    2580 scope.go:117] "RemoveContainer" containerID="784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0"
Jul 15 15:28:59.962968 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:28:59.962949    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0\": container with ID starting with 784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0 not found: ID does not exist" containerID="784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0"
Jul 15 15:28:59.963039 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.962977    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0"} err="failed to get container status \"784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0\": rpc error: code = NotFound desc = could not find container \"784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0\": container with ID starting with 784ca0f71066f84e4c55098b7272fa276ac31bc1e473585d9a76064f07fefaf0 not found: ID does not exist"
Jul 15 15:28:59.974316 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.974294    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b5b79d4b-ljcpz"]
Jul 15 15:28:59.981051 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:28:59.981029    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b5b79d4b-ljcpz"]
Jul 15 15:29:01.767463 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:01.767418    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ee8539-3140-42dc-b190-f294855304ca" path="/var/lib/kubelet/pods/13ee8539-3140-42dc-b190-f294855304ca/volumes"
Jul 15 15:29:34.241376 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.241339    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"]
Jul 15 15:29:34.241863 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.241684    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ee8539-3140-42dc-b190-f294855304ca" containerName="console"
Jul 15 15:29:34.241863 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.241695    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ee8539-3140-42dc-b190-f294855304ca" containerName="console"
Jul 15 15:29:34.241863 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.241778    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ee8539-3140-42dc-b190-f294855304ca" containerName="console"
Jul 15 15:29:34.244733 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.244716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.248792 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.248773    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 15:29:34.248915 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.248807    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j6nf4\""
Jul 15 15:29:34.249826 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.249804    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 15:29:34.257669 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.257650    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"]
Jul 15 15:29:34.332542 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.332507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlrcq\" (UniqueName: \"kubernetes.io/projected/4dd53334-7294-4789-8df6-1273d6477906-kube-api-access-qlrcq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.332702 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.332587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.332702 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.332617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.433860 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.433830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlrcq\" (UniqueName: \"kubernetes.io/projected/4dd53334-7294-4789-8df6-1273d6477906-kube-api-access-qlrcq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.433966 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.433912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.433966 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.433947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.434283 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.434263    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.434336 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.434321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.442762 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.442719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlrcq\" (UniqueName: \"kubernetes.io/projected/4dd53334-7294-4789-8df6-1273d6477906-kube-api-access-qlrcq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.553639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.553575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:34.681919 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:34.681891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"]
Jul 15 15:29:34.683846 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:29:34.683815    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd53334_7294_4789_8df6_1273d6477906.slice/crio-02f9c9b5ab250b819946cfffc8917028bb1a0889254e70caaad9afb1224e2a94 WatchSource:0}: Error finding container 02f9c9b5ab250b819946cfffc8917028bb1a0889254e70caaad9afb1224e2a94: Status 404 returned error can't find the container with id 02f9c9b5ab250b819946cfffc8917028bb1a0889254e70caaad9afb1224e2a94
Jul 15 15:29:35.089543 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:35.089509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t" event={"ID":"4dd53334-7294-4789-8df6-1273d6477906","Type":"ContainerStarted","Data":"02f9c9b5ab250b819946cfffc8917028bb1a0889254e70caaad9afb1224e2a94"}
Jul 15 15:29:35.604282 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:35.604255    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:29:35.605338 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:35.605316    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:29:35.609185 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:35.609162    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 15:29:42.122506 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:42.122460    2580 generic.go:356] "Generic (PLEG): container finished" podID="4dd53334-7294-4789-8df6-1273d6477906" containerID="dbb23fa976b0453672e4c566e4402ab2e52313df2967c5639911be1f868ca073" exitCode=0
Jul 15 15:29:42.122909 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:42.122524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t" event={"ID":"4dd53334-7294-4789-8df6-1273d6477906","Type":"ContainerDied","Data":"dbb23fa976b0453672e4c566e4402ab2e52313df2967c5639911be1f868ca073"}
Jul 15 15:29:42.123566 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:42.123549    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 15:29:44.132789 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:44.132736    2580 generic.go:356] "Generic (PLEG): container finished" podID="4dd53334-7294-4789-8df6-1273d6477906" containerID="de55f9ae56af632d743c2c44e0c221512c06736483a2447b4845ae661c42741f" exitCode=0
Jul 15 15:29:44.132789 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:44.132779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t" event={"ID":"4dd53334-7294-4789-8df6-1273d6477906","Type":"ContainerDied","Data":"de55f9ae56af632d743c2c44e0c221512c06736483a2447b4845ae661c42741f"}
Jul 15 15:29:51.166159 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:51.166084    2580 generic.go:356] "Generic (PLEG): container finished" podID="4dd53334-7294-4789-8df6-1273d6477906" containerID="f0d207e43f59072740bde0dd87d903fb4f44063b58e5ae47d58a5b2168589c02" exitCode=0
Jul 15 15:29:51.166159 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:51.166120    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t" event={"ID":"4dd53334-7294-4789-8df6-1273d6477906","Type":"ContainerDied","Data":"f0d207e43f59072740bde0dd87d903fb4f44063b58e5ae47d58a5b2168589c02"}
Jul 15 15:29:52.291872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.291846    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:52.397344 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.397314    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-util\") pod \"4dd53334-7294-4789-8df6-1273d6477906\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") "
Jul 15 15:29:52.397470 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.397397    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-bundle\") pod \"4dd53334-7294-4789-8df6-1273d6477906\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") "
Jul 15 15:29:52.397470 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.397427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlrcq\" (UniqueName: \"kubernetes.io/projected/4dd53334-7294-4789-8df6-1273d6477906-kube-api-access-qlrcq\") pod \"4dd53334-7294-4789-8df6-1273d6477906\" (UID: \"4dd53334-7294-4789-8df6-1273d6477906\") "
Jul 15 15:29:52.398175 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.398146    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-bundle" (OuterVolumeSpecName: "bundle") pod "4dd53334-7294-4789-8df6-1273d6477906" (UID: "4dd53334-7294-4789-8df6-1273d6477906"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:29:52.399604 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.399583    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd53334-7294-4789-8df6-1273d6477906-kube-api-access-qlrcq" (OuterVolumeSpecName: "kube-api-access-qlrcq") pod "4dd53334-7294-4789-8df6-1273d6477906" (UID: "4dd53334-7294-4789-8df6-1273d6477906"). InnerVolumeSpecName "kube-api-access-qlrcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:29:52.402052 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.402026    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-util" (OuterVolumeSpecName: "util") pod "4dd53334-7294-4789-8df6-1273d6477906" (UID: "4dd53334-7294-4789-8df6-1273d6477906"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:29:52.498221 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.498160    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:29:52.498221 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.498183    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlrcq\" (UniqueName: \"kubernetes.io/projected/4dd53334-7294-4789-8df6-1273d6477906-kube-api-access-qlrcq\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:29:52.498221 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:52.498192    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4dd53334-7294-4789-8df6-1273d6477906-util\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:29:53.176703 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:53.176662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t" event={"ID":"4dd53334-7294-4789-8df6-1273d6477906","Type":"ContainerDied","Data":"02f9c9b5ab250b819946cfffc8917028bb1a0889254e70caaad9afb1224e2a94"}
Jul 15 15:29:53.176703 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:53.176692    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hwl5t"
Jul 15 15:29:53.176921 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:53.176720    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02f9c9b5ab250b819946cfffc8917028bb1a0889254e70caaad9afb1224e2a94"
Jul 15 15:29:57.237292 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237254    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"]
Jul 15 15:29:57.237659 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237556    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd53334-7294-4789-8df6-1273d6477906" containerName="extract"
Jul 15 15:29:57.237659 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237567    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd53334-7294-4789-8df6-1273d6477906" containerName="extract"
Jul 15 15:29:57.237659 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237576    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd53334-7294-4789-8df6-1273d6477906" containerName="pull"
Jul 15 15:29:57.237659 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237582    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd53334-7294-4789-8df6-1273d6477906" containerName="pull"
Jul 15 15:29:57.237659 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237589    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd53334-7294-4789-8df6-1273d6477906" containerName="util"
Jul 15 15:29:57.237659 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237595    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd53334-7294-4789-8df6-1273d6477906" containerName="util"
Jul 15 15:29:57.237869 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.237663    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dd53334-7294-4789-8df6-1273d6477906" containerName="extract"
Jul 15 15:29:57.291388 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.291361    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"]
Jul 15 15:29:57.291521 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.291455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.295657 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.295635    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 15 15:29:57.296709 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.296674    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4qjrv\""
Jul 15 15:29:57.301170 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.301154    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 15 15:29:57.335352 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.335328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqs86\" (UniqueName: \"kubernetes.io/projected/cef01fb5-7d0b-4cc2-9530-851c8d9a8e63-kube-api-access-wqs86\") pod \"cert-manager-operator-controller-manager-55c444bf86-q5rj8\" (UID: \"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.335454 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.335399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cef01fb5-7d0b-4cc2-9530-851c8d9a8e63-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q5rj8\" (UID: \"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.436098 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.436071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqs86\" (UniqueName: \"kubernetes.io/projected/cef01fb5-7d0b-4cc2-9530-851c8d9a8e63-kube-api-access-wqs86\") pod \"cert-manager-operator-controller-manager-55c444bf86-q5rj8\" (UID: \"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.436228 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.436141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cef01fb5-7d0b-4cc2-9530-851c8d9a8e63-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q5rj8\" (UID: \"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.436462 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.436447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cef01fb5-7d0b-4cc2-9530-851c8d9a8e63-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q5rj8\" (UID: \"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.447664 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.447635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqs86\" (UniqueName: \"kubernetes.io/projected/cef01fb5-7d0b-4cc2-9530-851c8d9a8e63-kube-api-access-wqs86\") pod \"cert-manager-operator-controller-manager-55c444bf86-q5rj8\" (UID: \"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.600481 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.600454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"
Jul 15 15:29:57.740332 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:57.740302    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8"]
Jul 15 15:29:57.744725 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:29:57.744700    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef01fb5_7d0b_4cc2_9530_851c8d9a8e63.slice/crio-9a834244eb60dd5c34d790461cc7cb4e35ff8fd2e7734978433314470443188c WatchSource:0}: Error finding container 9a834244eb60dd5c34d790461cc7cb4e35ff8fd2e7734978433314470443188c: Status 404 returned error can't find the container with id 9a834244eb60dd5c34d790461cc7cb4e35ff8fd2e7734978433314470443188c
Jul 15 15:29:58.198021 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:29:58.197989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8" event={"ID":"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63","Type":"ContainerStarted","Data":"9a834244eb60dd5c34d790461cc7cb4e35ff8fd2e7734978433314470443188c"}
Jul 15 15:30:00.209194 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:00.209161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8" event={"ID":"cef01fb5-7d0b-4cc2-9530-851c8d9a8e63","Type":"ContainerStarted","Data":"6493949c85b3ed6f173d203c00a6976d1676cd981d43e76972a76efb5464bdad"}
Jul 15 15:30:00.242711 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:00.242667    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q5rj8" podStartSLOduration=0.943262961 podStartE2EDuration="3.242651665s" totalImagesPullingTime="2.299388704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:30:00.241509454 +0000 UTC m=+325.042969273" watchObservedRunningTime="2026-07-15 15:30:00.242651665 +0000 UTC m=+325.044111479"
Jul 15 15:30:01.348962 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.348928    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"]
Jul 15 15:30:01.352089 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.352072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.354895 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.354872    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j6nf4\""
Jul 15 15:30:01.355369 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.355349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 15:30:01.355883 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.355867    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 15:30:01.360797 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.360776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.360904 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.360820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xz9p\" (UniqueName: \"kubernetes.io/projected/76051369-e662-47af-9345-32bafe84df0e-kube-api-access-5xz9p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.360904 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.360847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.367537 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.367515    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"]
Jul 15 15:30:01.461309 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.461272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.461494 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.461322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xz9p\" (UniqueName: \"kubernetes.io/projected/76051369-e662-47af-9345-32bafe84df0e-kube-api-access-5xz9p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.461494 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.461346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.461700 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.461677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.461801 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.461719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.474289 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.474262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xz9p\" (UniqueName: \"kubernetes.io/projected/76051369-e662-47af-9345-32bafe84df0e-kube-api-access-5xz9p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.661691 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.661608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:01.790485 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:01.790462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"]
Jul 15 15:30:01.792940 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:30:01.792916    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76051369_e662_47af_9345_32bafe84df0e.slice/crio-bae85018955fd6c41b45aa8e76844bf8eb0ae5124d9bf027dafb57dded0a3c2b WatchSource:0}: Error finding container bae85018955fd6c41b45aa8e76844bf8eb0ae5124d9bf027dafb57dded0a3c2b: Status 404 returned error can't find the container with id bae85018955fd6c41b45aa8e76844bf8eb0ae5124d9bf027dafb57dded0a3c2b
Jul 15 15:30:02.066747 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.066676    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8ttzx"]
Jul 15 15:30:02.068873 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.068857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.072009 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.071982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hds52\""
Jul 15 15:30:02.072297 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.072282    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 15:30:02.072606 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.072593    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 15:30:02.083391 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.083366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8ttzx"]
Jul 15 15:30:02.166146 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.166117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn28g\" (UniqueName: \"kubernetes.io/projected/bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874-kube-api-access-tn28g\") pod \"cert-manager-webhook-6486ffd997-8ttzx\" (UID: \"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.166265 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.166194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8ttzx\" (UID: \"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.224221 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.224190    2580 generic.go:356] "Generic (PLEG): container finished" podID="76051369-e662-47af-9345-32bafe84df0e" containerID="96f924e83c3296fd3ebb6b6b033b35c38be5d9f227eb60d79771ef68642a9a1e" exitCode=0
Jul 15 15:30:02.224347 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.224279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" event={"ID":"76051369-e662-47af-9345-32bafe84df0e","Type":"ContainerDied","Data":"96f924e83c3296fd3ebb6b6b033b35c38be5d9f227eb60d79771ef68642a9a1e"}
Jul 15 15:30:02.224347 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.224333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" event={"ID":"76051369-e662-47af-9345-32bafe84df0e","Type":"ContainerStarted","Data":"bae85018955fd6c41b45aa8e76844bf8eb0ae5124d9bf027dafb57dded0a3c2b"}
Jul 15 15:30:02.267226 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.267205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn28g\" (UniqueName: \"kubernetes.io/projected/bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874-kube-api-access-tn28g\") pod \"cert-manager-webhook-6486ffd997-8ttzx\" (UID: \"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.267337 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.267282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8ttzx\" (UID: \"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.278924 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.278895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8ttzx\" (UID: \"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.279033 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.279019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn28g\" (UniqueName: \"kubernetes.io/projected/bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874-kube-api-access-tn28g\") pod \"cert-manager-webhook-6486ffd997-8ttzx\" (UID: \"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.389456 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.389427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:02.513680 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:02.513646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8ttzx"]
Jul 15 15:30:02.516005 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:30:02.515982    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbea17ba2_3e60_4bc1_a3b5_78d7fc5c9874.slice/crio-89caa01fcc84aec3d867b0fee079d81a5110266ba56c8bf739daccf722919b5a WatchSource:0}: Error finding container 89caa01fcc84aec3d867b0fee079d81a5110266ba56c8bf739daccf722919b5a: Status 404 returned error can't find the container with id 89caa01fcc84aec3d867b0fee079d81a5110266ba56c8bf739daccf722919b5a
Jul 15 15:30:03.231054 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.231018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx" event={"ID":"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874","Type":"ContainerStarted","Data":"89caa01fcc84aec3d867b0fee079d81a5110266ba56c8bf739daccf722919b5a"}
Jul 15 15:30:03.742311 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.742274    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"]
Jul 15 15:30:03.745423 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.745405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:03.747795 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.747775    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dvcxp\""
Jul 15 15:30:03.755118 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.755100    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"]
Jul 15 15:30:03.780790 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.780743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d9e9278-b742-4242-83ab-2338a156d030-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9hnmm\" (UID: \"7d9e9278-b742-4242-83ab-2338a156d030\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:03.780889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.780804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbs6g\" (UniqueName: \"kubernetes.io/projected/7d9e9278-b742-4242-83ab-2338a156d030-kube-api-access-vbs6g\") pod \"cert-manager-cainjector-559cdd8c87-9hnmm\" (UID: \"7d9e9278-b742-4242-83ab-2338a156d030\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:03.882050 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.882024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d9e9278-b742-4242-83ab-2338a156d030-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9hnmm\" (UID: \"7d9e9278-b742-4242-83ab-2338a156d030\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:03.882138 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.882060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbs6g\" (UniqueName: \"kubernetes.io/projected/7d9e9278-b742-4242-83ab-2338a156d030-kube-api-access-vbs6g\") pod \"cert-manager-cainjector-559cdd8c87-9hnmm\" (UID: \"7d9e9278-b742-4242-83ab-2338a156d030\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:03.890547 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.890521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d9e9278-b742-4242-83ab-2338a156d030-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9hnmm\" (UID: \"7d9e9278-b742-4242-83ab-2338a156d030\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:03.890637 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:03.890600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbs6g\" (UniqueName: \"kubernetes.io/projected/7d9e9278-b742-4242-83ab-2338a156d030-kube-api-access-vbs6g\") pod \"cert-manager-cainjector-559cdd8c87-9hnmm\" (UID: \"7d9e9278-b742-4242-83ab-2338a156d030\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:04.055181 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:04.055118    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"
Jul 15 15:30:04.181992 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:04.181969    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm"]
Jul 15 15:30:04.184033 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:30:04.184003    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d9e9278_b742_4242_83ab_2338a156d030.slice/crio-6ed709e0cc244a2c0e4b419ec39e0c54cd06571114b6ba5b42fd0c059b5c2af4 WatchSource:0}: Error finding container 6ed709e0cc244a2c0e4b419ec39e0c54cd06571114b6ba5b42fd0c059b5c2af4: Status 404 returned error can't find the container with id 6ed709e0cc244a2c0e4b419ec39e0c54cd06571114b6ba5b42fd0c059b5c2af4
Jul 15 15:30:04.236719 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:04.236692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm" event={"ID":"7d9e9278-b742-4242-83ab-2338a156d030","Type":"ContainerStarted","Data":"6ed709e0cc244a2c0e4b419ec39e0c54cd06571114b6ba5b42fd0c059b5c2af4"}
Jul 15 15:30:06.249892 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:06.249864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" event={"ID":"76051369-e662-47af-9345-32bafe84df0e","Type":"ContainerStarted","Data":"fa50e5058f00339201a0f39be07c597ddef1f6028386f743ac37377ed75c76b8"}
Jul 15 15:30:07.257813 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:07.257773    2580 generic.go:356] "Generic (PLEG): container finished" podID="76051369-e662-47af-9345-32bafe84df0e" containerID="fa50e5058f00339201a0f39be07c597ddef1f6028386f743ac37377ed75c76b8" exitCode=0
Jul 15 15:30:07.258269 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:07.257853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" event={"ID":"76051369-e662-47af-9345-32bafe84df0e","Type":"ContainerDied","Data":"fa50e5058f00339201a0f39be07c597ddef1f6028386f743ac37377ed75c76b8"}
Jul 15 15:30:08.266009 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:08.265969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" event={"ID":"76051369-e662-47af-9345-32bafe84df0e","Type":"ContainerStarted","Data":"4173fc7b5b818d1609f8ff2caf156c052802113800f6da0a37a6bb24d733c05f"}
Jul 15 15:30:08.288217 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:08.288161    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" podStartSLOduration=3.352594313 podStartE2EDuration="7.288141765s" totalImagesPullingTime="3.935547452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:30:08.286083419 +0000 UTC m=+333.087543250" watchObservedRunningTime="2026-07-15 15:30:08.288141765 +0000 UTC m=+333.089601580"
Jul 15 15:30:09.273061 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:09.273031    2580 generic.go:356] "Generic (PLEG): container finished" podID="76051369-e662-47af-9345-32bafe84df0e" containerID="4173fc7b5b818d1609f8ff2caf156c052802113800f6da0a37a6bb24d733c05f" exitCode=0
Jul 15 15:30:09.273408 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:09.273103    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" event={"ID":"76051369-e662-47af-9345-32bafe84df0e","Type":"ContainerDied","Data":"4173fc7b5b818d1609f8ff2caf156c052802113800f6da0a37a6bb24d733c05f"}
Jul 15 15:30:09.274502 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:09.274473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx" event={"ID":"bea17ba2-3e60-4bc1-a3b5-78d7fc5c9874","Type":"ContainerStarted","Data":"3a19322117597b5bda39b107bd2ec2e1db0f440b95a2419598b58fd898f7917e"}
Jul 15 15:30:09.274585 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:09.274525    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:09.275918 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:09.275897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm" event={"ID":"7d9e9278-b742-4242-83ab-2338a156d030","Type":"ContainerStarted","Data":"56e064eba631cef5f461b153b5088074899a27f23b624f52a00cb8e21d420c20"}
Jul 15 15:30:09.308655 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:09.308601    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hnmm" podStartSLOduration=2.044234448 podStartE2EDuration="6.30858746s" totalImagesPullingTime="4.264353012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:30:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:30:09.3077703 +0000 UTC m=+334.109230136" watchObservedRunningTime="2026-07-15 15:30:09.30858746 +0000 UTC m=+334.110047274"
Jul 15 15:30:09.333589 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:09.333534    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx" podStartSLOduration=1.4096924849999999 podStartE2EDuration="7.333518914s" totalImagesPullingTime="5.923826429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:30:09.332184605 +0000 UTC m=+334.133644419" watchObservedRunningTime="2026-07-15 15:30:09.333518914 +0000 UTC m=+334.134978728"
Jul 15 15:30:10.405742 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.405719    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:10.444473 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.444444    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-bundle\") pod \"76051369-e662-47af-9345-32bafe84df0e\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") "
Jul 15 15:30:10.444604 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.444482    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-util\") pod \"76051369-e662-47af-9345-32bafe84df0e\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") "
Jul 15 15:30:10.444604 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.444534    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xz9p\" (UniqueName: \"kubernetes.io/projected/76051369-e662-47af-9345-32bafe84df0e-kube-api-access-5xz9p\") pod \"76051369-e662-47af-9345-32bafe84df0e\" (UID: \"76051369-e662-47af-9345-32bafe84df0e\") "
Jul 15 15:30:10.444859 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.444829    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-bundle" (OuterVolumeSpecName: "bundle") pod "76051369-e662-47af-9345-32bafe84df0e" (UID: "76051369-e662-47af-9345-32bafe84df0e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:30:10.446625 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.446600    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76051369-e662-47af-9345-32bafe84df0e-kube-api-access-5xz9p" (OuterVolumeSpecName: "kube-api-access-5xz9p") pod "76051369-e662-47af-9345-32bafe84df0e" (UID: "76051369-e662-47af-9345-32bafe84df0e"). InnerVolumeSpecName "kube-api-access-5xz9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:30:10.448947 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.448912    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-util" (OuterVolumeSpecName: "util") pod "76051369-e662-47af-9345-32bafe84df0e" (UID: "76051369-e662-47af-9345-32bafe84df0e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:30:10.545470 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.545403    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:30:10.545470 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.545437    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76051369-e662-47af-9345-32bafe84df0e-util\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:30:10.545470 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:10.545446    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xz9p\" (UniqueName: \"kubernetes.io/projected/76051369-e662-47af-9345-32bafe84df0e-kube-api-access-5xz9p\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:30:11.286435 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.286407    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4"
Jul 15 15:30:11.286597 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.286407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mz4" event={"ID":"76051369-e662-47af-9345-32bafe84df0e","Type":"ContainerDied","Data":"bae85018955fd6c41b45aa8e76844bf8eb0ae5124d9bf027dafb57dded0a3c2b"}
Jul 15 15:30:11.286597 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.286524    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bae85018955fd6c41b45aa8e76844bf8eb0ae5124d9bf027dafb57dded0a3c2b"
Jul 15 15:30:11.820859 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.820828    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qhjd8"]
Jul 15 15:30:11.821214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.821135    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76051369-e662-47af-9345-32bafe84df0e" containerName="util"
Jul 15 15:30:11.821214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.821147    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="76051369-e662-47af-9345-32bafe84df0e" containerName="util"
Jul 15 15:30:11.821214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.821166    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76051369-e662-47af-9345-32bafe84df0e" containerName="pull"
Jul 15 15:30:11.821214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.821173    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="76051369-e662-47af-9345-32bafe84df0e" containerName="pull"
Jul 15 15:30:11.821214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.821179    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76051369-e662-47af-9345-32bafe84df0e" containerName="extract"
Jul 15 15:30:11.821214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.821185    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="76051369-e662-47af-9345-32bafe84df0e" containerName="extract"
Jul 15 15:30:11.821393 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.821246    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="76051369-e662-47af-9345-32bafe84df0e" containerName="extract"
Jul 15 15:30:11.823434 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.823417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:11.826067 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.826044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wgn5m\""
Jul 15 15:30:11.834477 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.834456    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qhjd8"]
Jul 15 15:30:11.855570 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.855548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3915725-09f2-4fc9-82c1-c08c4e248320-bound-sa-token\") pod \"cert-manager-55b4d954d5-qhjd8\" (UID: \"a3915725-09f2-4fc9-82c1-c08c4e248320\") " pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:11.855672 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.855587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj9hn\" (UniqueName: \"kubernetes.io/projected/a3915725-09f2-4fc9-82c1-c08c4e248320-kube-api-access-wj9hn\") pod \"cert-manager-55b4d954d5-qhjd8\" (UID: \"a3915725-09f2-4fc9-82c1-c08c4e248320\") " pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:11.956393 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.956366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3915725-09f2-4fc9-82c1-c08c4e248320-bound-sa-token\") pod \"cert-manager-55b4d954d5-qhjd8\" (UID: \"a3915725-09f2-4fc9-82c1-c08c4e248320\") " pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:11.956489 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.956406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj9hn\" (UniqueName: \"kubernetes.io/projected/a3915725-09f2-4fc9-82c1-c08c4e248320-kube-api-access-wj9hn\") pod \"cert-manager-55b4d954d5-qhjd8\" (UID: \"a3915725-09f2-4fc9-82c1-c08c4e248320\") " pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:11.965926 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.965902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3915725-09f2-4fc9-82c1-c08c4e248320-bound-sa-token\") pod \"cert-manager-55b4d954d5-qhjd8\" (UID: \"a3915725-09f2-4fc9-82c1-c08c4e248320\") " pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:11.966200 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:11.966178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj9hn\" (UniqueName: \"kubernetes.io/projected/a3915725-09f2-4fc9-82c1-c08c4e248320-kube-api-access-wj9hn\") pod \"cert-manager-55b4d954d5-qhjd8\" (UID: \"a3915725-09f2-4fc9-82c1-c08c4e248320\") " pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:12.132662 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:12.132595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qhjd8"
Jul 15 15:30:12.257828 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:12.257804    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qhjd8"]
Jul 15 15:30:12.259921 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:30:12.259892    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3915725_09f2_4fc9_82c1_c08c4e248320.slice/crio-f553ab839d883f4b48464d942d53b76e44e8e902e69826c543863b627192f4d4 WatchSource:0}: Error finding container f553ab839d883f4b48464d942d53b76e44e8e902e69826c543863b627192f4d4: Status 404 returned error can't find the container with id f553ab839d883f4b48464d942d53b76e44e8e902e69826c543863b627192f4d4
Jul 15 15:30:12.291284 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:12.291261    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qhjd8" event={"ID":"a3915725-09f2-4fc9-82c1-c08c4e248320","Type":"ContainerStarted","Data":"f553ab839d883f4b48464d942d53b76e44e8e902e69826c543863b627192f4d4"}
Jul 15 15:30:13.299734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:13.299701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qhjd8" event={"ID":"a3915725-09f2-4fc9-82c1-c08c4e248320","Type":"ContainerStarted","Data":"1991bf22c16e4c4993b2b124dee49b2dbc0348195efffe6f5494f28dae22c4f7"}
Jul 15 15:30:13.318847 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:13.318797    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qhjd8" podStartSLOduration=2.318780325 podStartE2EDuration="2.318780325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:30:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:30:13.317221614 +0000 UTC m=+338.118681428" watchObservedRunningTime="2026-07-15 15:30:13.318780325 +0000 UTC m=+338.120240136"
Jul 15 15:30:15.282618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.282590    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8ttzx"
Jul 15 15:30:15.548798 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.548702    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"]
Jul 15 15:30:15.551169 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.551152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.554182 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.554158    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5cwnj\""
Jul 15 15:30:15.554319 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.554255    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 15:30:15.554319 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.554271    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 15:30:15.563993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.563969    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"]
Jul 15 15:30:15.588552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.588526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/044c6944-9a79-4c7b-8b44-0261ae4f50e4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xm4l4\" (UID: \"044c6944-9a79-4c7b-8b44-0261ae4f50e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.588673 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.588597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmx4\" (UniqueName: \"kubernetes.io/projected/044c6944-9a79-4c7b-8b44-0261ae4f50e4-kube-api-access-wnmx4\") pod \"openshift-lws-operator-fd8ccff4c-xm4l4\" (UID: \"044c6944-9a79-4c7b-8b44-0261ae4f50e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.689243 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.689207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/044c6944-9a79-4c7b-8b44-0261ae4f50e4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xm4l4\" (UID: \"044c6944-9a79-4c7b-8b44-0261ae4f50e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.689392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.689271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmx4\" (UniqueName: \"kubernetes.io/projected/044c6944-9a79-4c7b-8b44-0261ae4f50e4-kube-api-access-wnmx4\") pod \"openshift-lws-operator-fd8ccff4c-xm4l4\" (UID: \"044c6944-9a79-4c7b-8b44-0261ae4f50e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.689592 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.689572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/044c6944-9a79-4c7b-8b44-0261ae4f50e4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xm4l4\" (UID: \"044c6944-9a79-4c7b-8b44-0261ae4f50e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.700091 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.700062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmx4\" (UniqueName: \"kubernetes.io/projected/044c6944-9a79-4c7b-8b44-0261ae4f50e4-kube-api-access-wnmx4\") pod \"openshift-lws-operator-fd8ccff4c-xm4l4\" (UID: \"044c6944-9a79-4c7b-8b44-0261ae4f50e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.860482 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.860454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"
Jul 15 15:30:15.989479 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:15.989452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4"]
Jul 15 15:30:15.992058 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:30:15.992023    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod044c6944_9a79_4c7b_8b44_0261ae4f50e4.slice/crio-fde56bed6a83e7e44f3b4df990f697cd055c8ffde6b2da0e5aa020324d93f1cd WatchSource:0}: Error finding container fde56bed6a83e7e44f3b4df990f697cd055c8ffde6b2da0e5aa020324d93f1cd: Status 404 returned error can't find the container with id fde56bed6a83e7e44f3b4df990f697cd055c8ffde6b2da0e5aa020324d93f1cd
Jul 15 15:30:16.314730 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:16.314652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4" event={"ID":"044c6944-9a79-4c7b-8b44-0261ae4f50e4","Type":"ContainerStarted","Data":"fde56bed6a83e7e44f3b4df990f697cd055c8ffde6b2da0e5aa020324d93f1cd"}
Jul 15 15:30:19.330271 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:19.330228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4" event={"ID":"044c6944-9a79-4c7b-8b44-0261ae4f50e4","Type":"ContainerStarted","Data":"fb55d24caf6fe20a6075bb0a23df5d64ae0384266fffa95fc534b18bf04fa0e8"}
Jul 15 15:30:22.045178 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.045127    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xm4l4" podStartSLOduration=4.255501815 podStartE2EDuration="7.045110716s" totalImagesPullingTime="2.789608901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:30:19.354281048 +0000 UTC m=+344.155740869" watchObservedRunningTime="2026-07-15 15:30:22.045110716 +0000 UTC m=+346.846570532"
Jul 15 15:30:22.045582 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.045368    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"]
Jul 15 15:30:22.060584 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.060554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"]
Jul 15 15:30:22.060711 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.060648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.063351 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.063320    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 15 15:30:22.063453 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.063365    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j6nf4\""
Jul 15 15:30:22.063453 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.063372    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 15 15:30:22.149322 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.149290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.149437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.149334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.149437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.149389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dg6c\" (UniqueName: \"kubernetes.io/projected/941863de-dbbf-41a3-8ba3-6153d9fd6393-kube-api-access-6dg6c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.250582 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.250554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.250717 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.250594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.250717 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.250621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dg6c\" (UniqueName: \"kubernetes.io/projected/941863de-dbbf-41a3-8ba3-6153d9fd6393-kube-api-access-6dg6c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.250966 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.250944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.251041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.250971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.259007 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.258984    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dg6c\" (UniqueName: \"kubernetes.io/projected/941863de-dbbf-41a3-8ba3-6153d9fd6393-kube-api-access-6dg6c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.371283 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.371256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:22.522998 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:22.522975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"]
Jul 15 15:30:22.524855 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:30:22.524829    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941863de_dbbf_41a3_8ba3_6153d9fd6393.slice/crio-daafd7354c5bac0301b13c491a2e8feaa2424489bce30c2ce30c0c79349f24ed WatchSource:0}: Error finding container daafd7354c5bac0301b13c491a2e8feaa2424489bce30c2ce30c0c79349f24ed: Status 404 returned error can't find the container with id daafd7354c5bac0301b13c491a2e8feaa2424489bce30c2ce30c0c79349f24ed
Jul 15 15:30:22.672594 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:30:22.672561    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941863de_dbbf_41a3_8ba3_6153d9fd6393.slice/crio-0d7408fcc31a59ad96a9e1cec510a41d20b54fedd64de60a8a589aee75337078.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941863de_dbbf_41a3_8ba3_6153d9fd6393.slice/crio-conmon-0d7408fcc31a59ad96a9e1cec510a41d20b54fedd64de60a8a589aee75337078.scope\": RecentStats: unable to find data in memory cache]"
Jul 15 15:30:23.349132 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:23.349100    2580 generic.go:356] "Generic (PLEG): container finished" podID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerID="0d7408fcc31a59ad96a9e1cec510a41d20b54fedd64de60a8a589aee75337078" exitCode=0
Jul 15 15:30:23.349488 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:23.349186    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf" event={"ID":"941863de-dbbf-41a3-8ba3-6153d9fd6393","Type":"ContainerDied","Data":"0d7408fcc31a59ad96a9e1cec510a41d20b54fedd64de60a8a589aee75337078"}
Jul 15 15:30:23.349488 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:23.349227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf" event={"ID":"941863de-dbbf-41a3-8ba3-6153d9fd6393","Type":"ContainerStarted","Data":"daafd7354c5bac0301b13c491a2e8feaa2424489bce30c2ce30c0c79349f24ed"}
Jul 15 15:30:24.356057 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:24.356025    2580 generic.go:356] "Generic (PLEG): container finished" podID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerID="32142f1c5e58940fea5914734315ce81c5e863af00750a9cf3755f030032c281" exitCode=0
Jul 15 15:30:24.356430 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:24.356074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf" event={"ID":"941863de-dbbf-41a3-8ba3-6153d9fd6393","Type":"ContainerDied","Data":"32142f1c5e58940fea5914734315ce81c5e863af00750a9cf3755f030032c281"}
Jul 15 15:30:25.363201 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:25.363160    2580 generic.go:356] "Generic (PLEG): container finished" podID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerID="637488ee89e4d39530c7df93d68bb9c634be92d02134f9bdb468b2c62646b6f8" exitCode=0
Jul 15 15:30:25.363563 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:25.363205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf" event={"ID":"941863de-dbbf-41a3-8ba3-6153d9fd6393","Type":"ContainerDied","Data":"637488ee89e4d39530c7df93d68bb9c634be92d02134f9bdb468b2c62646b6f8"}
Jul 15 15:30:26.496295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.496271    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:26.586965 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.586937    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-bundle\") pod \"941863de-dbbf-41a3-8ba3-6153d9fd6393\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") "
Jul 15 15:30:26.587118 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.586989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-util\") pod \"941863de-dbbf-41a3-8ba3-6153d9fd6393\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") "
Jul 15 15:30:26.587118 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.587077    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dg6c\" (UniqueName: \"kubernetes.io/projected/941863de-dbbf-41a3-8ba3-6153d9fd6393-kube-api-access-6dg6c\") pod \"941863de-dbbf-41a3-8ba3-6153d9fd6393\" (UID: \"941863de-dbbf-41a3-8ba3-6153d9fd6393\") "
Jul 15 15:30:26.587647 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.587603    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-bundle" (OuterVolumeSpecName: "bundle") pod "941863de-dbbf-41a3-8ba3-6153d9fd6393" (UID: "941863de-dbbf-41a3-8ba3-6153d9fd6393"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:30:26.589063 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.589038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941863de-dbbf-41a3-8ba3-6153d9fd6393-kube-api-access-6dg6c" (OuterVolumeSpecName: "kube-api-access-6dg6c") pod "941863de-dbbf-41a3-8ba3-6153d9fd6393" (UID: "941863de-dbbf-41a3-8ba3-6153d9fd6393"). InnerVolumeSpecName "kube-api-access-6dg6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:30:26.592466 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.592443    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-util" (OuterVolumeSpecName: "util") pod "941863de-dbbf-41a3-8ba3-6153d9fd6393" (UID: "941863de-dbbf-41a3-8ba3-6153d9fd6393"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:30:26.688111 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.688024    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dg6c\" (UniqueName: \"kubernetes.io/projected/941863de-dbbf-41a3-8ba3-6153d9fd6393-kube-api-access-6dg6c\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:30:26.688111 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.688055    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:30:26.688111 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:26.688067    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/941863de-dbbf-41a3-8ba3-6153d9fd6393-util\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:30:27.373782 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:27.373702    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf"
Jul 15 15:30:27.373782 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:27.373701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afrvlpf" event={"ID":"941863de-dbbf-41a3-8ba3-6153d9fd6393","Type":"ContainerDied","Data":"daafd7354c5bac0301b13c491a2e8feaa2424489bce30c2ce30c0c79349f24ed"}
Jul 15 15:30:27.373951 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:27.373806    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="daafd7354c5bac0301b13c491a2e8feaa2424489bce30c2ce30c0c79349f24ed"
Jul 15 15:30:39.105852 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.105822    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"]
Jul 15 15:30:39.106413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.106198    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerName="extract"
Jul 15 15:30:39.106413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.106212    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerName="extract"
Jul 15 15:30:39.106413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.106221    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerName="util"
Jul 15 15:30:39.106413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.106227    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerName="util"
Jul 15 15:30:39.106413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.106248    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerName="pull"
Jul 15 15:30:39.106413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.106253    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerName="pull"
Jul 15 15:30:39.106413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.106306    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="941863de-dbbf-41a3-8ba3-6153d9fd6393" containerName="extract"
Jul 15 15:30:39.151995 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.151962    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"]
Jul 15 15:30:39.152144 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.152085    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.155572 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.155549    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r5ptn\""
Jul 15 15:30:39.155714 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.155552    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 15:30:39.155947 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.155926    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 15:30:39.156071 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.155982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 15:30:39.156071 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.155984    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 15:30:39.303478 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.303444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f42580a0-134b-4913-9b8a-61f3b6a61485-manifests\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.303648 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.303491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh8tb\" (UniqueName: \"kubernetes.io/projected/f42580a0-134b-4913-9b8a-61f3b6a61485-kube-api-access-gh8tb\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.303648 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.303525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f42580a0-134b-4913-9b8a-61f3b6a61485-webhook-cert\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.303648 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.303571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f42580a0-134b-4913-9b8a-61f3b6a61485-apiservice-cert\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.303648 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.303612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f42580a0-134b-4913-9b8a-61f3b6a61485-tmp\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.404229 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.404149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f42580a0-134b-4913-9b8a-61f3b6a61485-manifests\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.404229 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.404197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh8tb\" (UniqueName: \"kubernetes.io/projected/f42580a0-134b-4913-9b8a-61f3b6a61485-kube-api-access-gh8tb\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.404229 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.404220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f42580a0-134b-4913-9b8a-61f3b6a61485-webhook-cert\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.404502 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.404245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f42580a0-134b-4913-9b8a-61f3b6a61485-apiservice-cert\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.404502 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.404284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f42580a0-134b-4913-9b8a-61f3b6a61485-tmp\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.404639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.404615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f42580a0-134b-4913-9b8a-61f3b6a61485-manifests\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.406583 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.406559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f42580a0-134b-4913-9b8a-61f3b6a61485-tmp\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.406818 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.406798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f42580a0-134b-4913-9b8a-61f3b6a61485-webhook-cert\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.406936 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.406912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f42580a0-134b-4913-9b8a-61f3b6a61485-apiservice-cert\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.419488 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.419464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh8tb\" (UniqueName: \"kubernetes.io/projected/f42580a0-134b-4913-9b8a-61f3b6a61485-kube-api-access-gh8tb\") pod \"opendatahub-operator-controller-manager-997df6bcb-twxtg\" (UID: \"f42580a0-134b-4913-9b8a-61f3b6a61485\") " pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.463865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.463838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:39.634185 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:39.634164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"]
Jul 15 15:30:39.636075 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:30:39.636045    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42580a0_134b_4913_9b8a_61f3b6a61485.slice/crio-39ccc41e89299817ed3a5f9a943cab20b4ee048640bc9cc5ad6c8a994e18d309 WatchSource:0}: Error finding container 39ccc41e89299817ed3a5f9a943cab20b4ee048640bc9cc5ad6c8a994e18d309: Status 404 returned error can't find the container with id 39ccc41e89299817ed3a5f9a943cab20b4ee048640bc9cc5ad6c8a994e18d309
Jul 15 15:30:40.438101 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:40.438060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg" event={"ID":"f42580a0-134b-4913-9b8a-61f3b6a61485","Type":"ContainerStarted","Data":"39ccc41e89299817ed3a5f9a943cab20b4ee048640bc9cc5ad6c8a994e18d309"}
Jul 15 15:30:43.460874 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:43.460833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg" event={"ID":"f42580a0-134b-4913-9b8a-61f3b6a61485","Type":"ContainerStarted","Data":"54af6aba20ab61f37b719304feb16ed17bbb296ca467bec28853539475d78c0c"}
Jul 15 15:30:44.467550 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:44.467509    2580 generic.go:356] "Generic (PLEG): container finished" podID="f42580a0-134b-4913-9b8a-61f3b6a61485" containerID="54af6aba20ab61f37b719304feb16ed17bbb296ca467bec28853539475d78c0c" exitCode=0
Jul 15 15:30:44.467972 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:44.467598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg" event={"ID":"f42580a0-134b-4913-9b8a-61f3b6a61485","Type":"ContainerDied","Data":"54af6aba20ab61f37b719304feb16ed17bbb296ca467bec28853539475d78c0c"}
Jul 15 15:30:45.474725 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:45.474688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg" event={"ID":"f42580a0-134b-4913-9b8a-61f3b6a61485","Type":"ContainerStarted","Data":"86a75a3bbd70b04bb427e02be5972b8f999475334c74061aa0ff3c2a0cbdc123"}
Jul 15 15:30:45.475178 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:45.474809    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:30:45.501517 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:45.501467    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg" podStartSLOduration=2.073455043 podStartE2EDuration="6.501451108s" totalImagesPullingTime="3.391974092s" totalInitContainerRuntime="1.036021973s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:30:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:30:45.498291262 +0000 UTC m=+370.299751076" watchObservedRunningTime="2026-07-15 15:30:45.501451108 +0000 UTC m=+370.302910932"
Jul 15 15:30:56.482304 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:30:56.482273    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-997df6bcb-twxtg"
Jul 15 15:31:09.422366 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.422330    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"]
Jul 15 15:31:09.427065 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.427044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.430949 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.430919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xbblp\""
Jul 15 15:31:09.431088 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.430932    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 15:31:09.431088 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.430972    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 15:31:09.431088 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.431016    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 15:31:09.439517 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.439493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"]
Jul 15 15:31:09.567709 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.567680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12dc964a-65ac-4793-bde3-52733e840edb-cert\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.567893 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.567715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nfs4\" (UniqueName: \"kubernetes.io/projected/12dc964a-65ac-4793-bde3-52733e840edb-kube-api-access-4nfs4\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.567893 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.567739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12dc964a-65ac-4793-bde3-52733e840edb-metrics-cert\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.567893 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.567837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12dc964a-65ac-4793-bde3-52733e840edb-manager-config\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.668739 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.668706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12dc964a-65ac-4793-bde3-52733e840edb-manager-config\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.668912 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.668791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12dc964a-65ac-4793-bde3-52733e840edb-cert\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.668912 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.668811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nfs4\" (UniqueName: \"kubernetes.io/projected/12dc964a-65ac-4793-bde3-52733e840edb-kube-api-access-4nfs4\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.668912 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.668829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12dc964a-65ac-4793-bde3-52733e840edb-metrics-cert\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.669345 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.669323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/12dc964a-65ac-4793-bde3-52733e840edb-manager-config\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.671237 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.671213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/12dc964a-65ac-4793-bde3-52733e840edb-cert\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.671318 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.671270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12dc964a-65ac-4793-bde3-52733e840edb-metrics-cert\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.682041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.681991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nfs4\" (UniqueName: \"kubernetes.io/projected/12dc964a-65ac-4793-bde3-52733e840edb-kube-api-access-4nfs4\") pod \"lws-controller-manager-6c5d886c95-4b4gn\" (UID: \"12dc964a-65ac-4793-bde3-52733e840edb\") " pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.737076 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.737033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:09.908716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:09.908689    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"]
Jul 15 15:31:09.911151 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:31:09.911119    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12dc964a_65ac_4793_bde3_52733e840edb.slice/crio-07f73cc9c9f00aaa9b20dc0b56160dc8f3b840580741c89fb90d51dc38b403dc WatchSource:0}: Error finding container 07f73cc9c9f00aaa9b20dc0b56160dc8f3b840580741c89fb90d51dc38b403dc: Status 404 returned error can't find the container with id 07f73cc9c9f00aaa9b20dc0b56160dc8f3b840580741c89fb90d51dc38b403dc
Jul 15 15:31:10.563694 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.563661    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"]
Jul 15 15:31:10.568065 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.568045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:10.572217 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.572191    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rq5pk\""
Jul 15 15:31:10.579130 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.579110    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 15 15:31:10.589265 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.589245    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"]
Jul 15 15:31:10.603922 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.603897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn" event={"ID":"12dc964a-65ac-4793-bde3-52733e840edb","Type":"ContainerStarted","Data":"07f73cc9c9f00aaa9b20dc0b56160dc8f3b840580741c89fb90d51dc38b403dc"}
Jul 15 15:31:10.678946 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.678921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx9sk\" (UniqueName: \"kubernetes.io/projected/b8330683-b064-4759-8568-4ebf931d0c21-kube-api-access-dx9sk\") pod \"llmisvc-controller-manager-777c6bd7d5-ktlm2\" (UID: \"b8330683-b064-4759-8568-4ebf931d0c21\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:10.679053 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.678964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8330683-b064-4759-8568-4ebf931d0c21-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktlm2\" (UID: \"b8330683-b064-4759-8568-4ebf931d0c21\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:10.779958 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.779919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx9sk\" (UniqueName: \"kubernetes.io/projected/b8330683-b064-4759-8568-4ebf931d0c21-kube-api-access-dx9sk\") pod \"llmisvc-controller-manager-777c6bd7d5-ktlm2\" (UID: \"b8330683-b064-4759-8568-4ebf931d0c21\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:10.780095 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.780000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8330683-b064-4759-8568-4ebf931d0c21-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktlm2\" (UID: \"b8330683-b064-4759-8568-4ebf931d0c21\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:10.780168 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:31:10.780141    2580 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 15 15:31:10.780221 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:31:10.780213    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8330683-b064-4759-8568-4ebf931d0c21-cert podName:b8330683-b064-4759-8568-4ebf931d0c21 nodeName:}" failed. No retries permitted until 2026-07-15 15:31:11.280193208 +0000 UTC m=+396.081653002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b8330683-b064-4759-8568-4ebf931d0c21-cert") pod "llmisvc-controller-manager-777c6bd7d5-ktlm2" (UID: "b8330683-b064-4759-8568-4ebf931d0c21") : secret "llmisvc-webhook-server-cert" not found
Jul 15 15:31:10.803629 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:10.803601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx9sk\" (UniqueName: \"kubernetes.io/projected/b8330683-b064-4759-8568-4ebf931d0c21-kube-api-access-dx9sk\") pod \"llmisvc-controller-manager-777c6bd7d5-ktlm2\" (UID: \"b8330683-b064-4759-8568-4ebf931d0c21\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:11.285734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:11.285695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8330683-b064-4759-8568-4ebf931d0c21-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktlm2\" (UID: \"b8330683-b064-4759-8568-4ebf931d0c21\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:11.288779 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:11.288732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8330683-b064-4759-8568-4ebf931d0c21-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktlm2\" (UID: \"b8330683-b064-4759-8568-4ebf931d0c21\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:11.478978 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:11.478944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:11.699117 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:11.699091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"]
Jul 15 15:31:11.701144 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:31:11.701120    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb8330683_b064_4759_8568_4ebf931d0c21.slice/crio-aaeff1be6da90cd3a11b1ab9b78704be985692d51b66611ecdf12767b270c853 WatchSource:0}: Error finding container aaeff1be6da90cd3a11b1ab9b78704be985692d51b66611ecdf12767b270c853: Status 404 returned error can't find the container with id aaeff1be6da90cd3a11b1ab9b78704be985692d51b66611ecdf12767b270c853
Jul 15 15:31:12.616252 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:12.616212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2" event={"ID":"b8330683-b064-4759-8568-4ebf931d0c21","Type":"ContainerStarted","Data":"aaeff1be6da90cd3a11b1ab9b78704be985692d51b66611ecdf12767b270c853"}
Jul 15 15:31:12.618469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:12.618438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn" event={"ID":"12dc964a-65ac-4793-bde3-52733e840edb","Type":"ContainerStarted","Data":"0a8825f779eba23bd43a3c07b31b529617642fc3aaf1026bcef6e0e15942b9d6"}
Jul 15 15:31:12.618640 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:12.618559    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:12.640692 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:12.640647    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn" podStartSLOduration=1.909540416 podStartE2EDuration="3.640630022s" totalImagesPullingTime="1.731089606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:31:12.638386929 +0000 UTC m=+397.439846744" watchObservedRunningTime="2026-07-15 15:31:12.640630022 +0000 UTC m=+397.442089835"
Jul 15 15:31:15.635877 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:15.635845    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2" event={"ID":"b8330683-b064-4759-8568-4ebf931d0c21","Type":"ContainerStarted","Data":"5134eae7770e4d046792f24924b4ad77ac316db9966fb3f985578b8dfda831f1"}
Jul 15 15:31:15.636254 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:15.635988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:31:15.654193 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:15.654146    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2" podStartSLOduration=2.269485844 podStartE2EDuration="5.654132415s" totalImagesPullingTime="3.384646571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:31:15.653098893 +0000 UTC m=+400.454558730" watchObservedRunningTime="2026-07-15 15:31:15.654132415 +0000 UTC m=+400.455592229"
Jul 15 15:31:23.625769 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:23.625718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c5d886c95-4b4gn"
Jul 15 15:31:46.643369 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:31:46.643333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktlm2"
Jul 15 15:32:16.441378 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.441346    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"]
Jul 15 15:32:16.444060 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.444040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.446854 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.446834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7n4mz\""
Jul 15 15:32:16.446970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.446834    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 15:32:16.446970 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.446835    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 15:32:16.453418 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.453398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"]
Jul 15 15:32:16.542160 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.542125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.542319 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.542209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z864t\" (UniqueName: \"kubernetes.io/projected/3fe1f4e4-3a2d-4173-9714-e239fb122f32-kube-api-access-z864t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.542362 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.542314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.643409 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.643377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z864t\" (UniqueName: \"kubernetes.io/projected/3fe1f4e4-3a2d-4173-9714-e239fb122f32-kube-api-access-z864t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.643576 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.643436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.643576 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.643477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.643864 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.643842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.643972 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.643880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.652238 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.652215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z864t\" (UniqueName: \"kubernetes.io/projected/3fe1f4e4-3a2d-4173-9714-e239fb122f32-kube-api-access-z864t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.754085 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.754015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:16.843082 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.843040    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"]
Jul 15 15:32:16.846316 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.846295    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:16.858533 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.858512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"]
Jul 15 15:32:16.893574 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.893552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"]
Jul 15 15:32:16.895210 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:32:16.895182    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe1f4e4_3a2d_4173_9714_e239fb122f32.slice/crio-24b41da3d1459da87eee6e3a5ce96757fa6efdc8f7bd1cfd17d38a814f7b83fe WatchSource:0}: Error finding container 24b41da3d1459da87eee6e3a5ce96757fa6efdc8f7bd1cfd17d38a814f7b83fe: Status 404 returned error can't find the container with id 24b41da3d1459da87eee6e3a5ce96757fa6efdc8f7bd1cfd17d38a814f7b83fe
Jul 15 15:32:16.926796 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.926771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z" event={"ID":"3fe1f4e4-3a2d-4173-9714-e239fb122f32","Type":"ContainerStarted","Data":"24b41da3d1459da87eee6e3a5ce96757fa6efdc8f7bd1cfd17d38a814f7b83fe"}
Jul 15 15:32:16.946068 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.946043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:16.946165 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.946108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg25j\" (UniqueName: \"kubernetes.io/projected/d158794c-7d7d-42c8-8364-72aac59b575b-kube-api-access-cg25j\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:16.946165 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:16.946150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.047308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.047269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg25j\" (UniqueName: \"kubernetes.io/projected/d158794c-7d7d-42c8-8364-72aac59b575b-kube-api-access-cg25j\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.047433 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.047334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.047495 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.047441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.047720 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.047697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.047845 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.047824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.057103 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.057081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg25j\" (UniqueName: \"kubernetes.io/projected/d158794c-7d7d-42c8-8364-72aac59b575b-kube-api-access-cg25j\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.157244 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.157215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:17.245586 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.245557    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"]
Jul 15 15:32:17.249320 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.249301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.256413 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.256388    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"]
Jul 15 15:32:17.287236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.287213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"]
Jul 15 15:32:17.289762 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:32:17.289729    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd158794c_7d7d_42c8_8364_72aac59b575b.slice/crio-07f6acc197f36505e32dfdac9b12df09eb0c931d6750ff9bcbb4abe6589d43b5 WatchSource:0}: Error finding container 07f6acc197f36505e32dfdac9b12df09eb0c931d6750ff9bcbb4abe6589d43b5: Status 404 returned error can't find the container with id 07f6acc197f36505e32dfdac9b12df09eb0c931d6750ff9bcbb4abe6589d43b5
Jul 15 15:32:17.349843 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.349820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.349986 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.349851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw4b6\" (UniqueName: \"kubernetes.io/projected/a42fccbc-1d50-4ae0-91a2-763b44861dd9-kube-api-access-zw4b6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.349986 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.349892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.451392 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.451364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.451783 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.451396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw4b6\" (UniqueName: \"kubernetes.io/projected/a42fccbc-1d50-4ae0-91a2-763b44861dd9-kube-api-access-zw4b6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.451783 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.451506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.451783 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.451723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.451947 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.451786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.460140 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.460116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw4b6\" (UniqueName: \"kubernetes.io/projected/a42fccbc-1d50-4ae0-91a2-763b44861dd9-kube-api-access-zw4b6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.561330 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.561273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:17.642267 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.642236    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"]
Jul 15 15:32:17.645846 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.645823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.654789 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.654743    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"]
Jul 15 15:32:17.694084 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.694064    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"]
Jul 15 15:32:17.695898 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:32:17.695873    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42fccbc_1d50_4ae0_91a2_763b44861dd9.slice/crio-de62ddb6a6e23e972c90c22384750d8198b71402454d86d866ee528f6758e64d WatchSource:0}: Error finding container de62ddb6a6e23e972c90c22384750d8198b71402454d86d866ee528f6758e64d: Status 404 returned error can't find the container with id de62ddb6a6e23e972c90c22384750d8198b71402454d86d866ee528f6758e64d
Jul 15 15:32:17.754193 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.754173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.754301 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.754200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.754301 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.754220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq675\" (UniqueName: \"kubernetes.io/projected/b46c748a-0c0e-45fb-8a85-0ba04cb78361-kube-api-access-hq675\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.855436 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.855406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.855579 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.855440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.855579 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.855461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq675\" (UniqueName: \"kubernetes.io/projected/b46c748a-0c0e-45fb-8a85-0ba04cb78361-kube-api-access-hq675\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.855845 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.855825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.855924 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.855861    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.863781 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.863738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq675\" (UniqueName: \"kubernetes.io/projected/b46c748a-0c0e-45fb-8a85-0ba04cb78361-kube-api-access-hq675\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:17.933601 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.933565    2580 generic.go:356] "Generic (PLEG): container finished" podID="d158794c-7d7d-42c8-8364-72aac59b575b" containerID="088867a291897cdbd08bbf7e92f3a132f73ecd3875e41d5120dc1f0b26598e7d" exitCode=0
Jul 15 15:32:17.933713 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.933649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc" event={"ID":"d158794c-7d7d-42c8-8364-72aac59b575b","Type":"ContainerDied","Data":"088867a291897cdbd08bbf7e92f3a132f73ecd3875e41d5120dc1f0b26598e7d"}
Jul 15 15:32:17.933793 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.933713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc" event={"ID":"d158794c-7d7d-42c8-8364-72aac59b575b","Type":"ContainerStarted","Data":"07f6acc197f36505e32dfdac9b12df09eb0c931d6750ff9bcbb4abe6589d43b5"}
Jul 15 15:32:17.935279 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.935253    2580 generic.go:356] "Generic (PLEG): container finished" podID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerID="005a89e8d814c9012efc7cd9c3ced073b6a9de3ad49c92ab7f375076645f37a4" exitCode=0
Jul 15 15:32:17.935376 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.935326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z" event={"ID":"3fe1f4e4-3a2d-4173-9714-e239fb122f32","Type":"ContainerDied","Data":"005a89e8d814c9012efc7cd9c3ced073b6a9de3ad49c92ab7f375076645f37a4"}
Jul 15 15:32:17.937023 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.936979    2580 generic.go:356] "Generic (PLEG): container finished" podID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerID="95f7b1497ffd84da542b5106f523d3015597a86d8c029c411dfe088847332096" exitCode=0
Jul 15 15:32:17.937023 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.937014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v" event={"ID":"a42fccbc-1d50-4ae0-91a2-763b44861dd9","Type":"ContainerDied","Data":"95f7b1497ffd84da542b5106f523d3015597a86d8c029c411dfe088847332096"}
Jul 15 15:32:17.937146 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.937052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v" event={"ID":"a42fccbc-1d50-4ae0-91a2-763b44861dd9","Type":"ContainerStarted","Data":"de62ddb6a6e23e972c90c22384750d8198b71402454d86d866ee528f6758e64d"}
Jul 15 15:32:17.975486 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:17.975464    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:18.126969 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.126865    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"]
Jul 15 15:32:18.129640 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:32:18.129611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb46c748a_0c0e_45fb_8a85_0ba04cb78361.slice/crio-ba2104fb3b1b6832ce03f037b22fa80c44546b6800fab68b9d174de98d903f39 WatchSource:0}: Error finding container ba2104fb3b1b6832ce03f037b22fa80c44546b6800fab68b9d174de98d903f39: Status 404 returned error can't find the container with id ba2104fb3b1b6832ce03f037b22fa80c44546b6800fab68b9d174de98d903f39
Jul 15 15:32:18.943654 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.943621    2580 generic.go:356] "Generic (PLEG): container finished" podID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerID="2310b6a43d738f1226b330f8956119f4048bd66e7c73a37eab078a15b2cfd147" exitCode=0
Jul 15 15:32:18.944120 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.943714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4" event={"ID":"b46c748a-0c0e-45fb-8a85-0ba04cb78361","Type":"ContainerDied","Data":"2310b6a43d738f1226b330f8956119f4048bd66e7c73a37eab078a15b2cfd147"}
Jul 15 15:32:18.944120 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.943799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4" event={"ID":"b46c748a-0c0e-45fb-8a85-0ba04cb78361","Type":"ContainerStarted","Data":"ba2104fb3b1b6832ce03f037b22fa80c44546b6800fab68b9d174de98d903f39"}
Jul 15 15:32:18.945528 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.945498    2580 generic.go:356] "Generic (PLEG): container finished" podID="d158794c-7d7d-42c8-8364-72aac59b575b" containerID="fdbb152609c859169fcad85769147213f5223448268240862e4c94c3bc4cf9da" exitCode=0
Jul 15 15:32:18.945633 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.945587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc" event={"ID":"d158794c-7d7d-42c8-8364-72aac59b575b","Type":"ContainerDied","Data":"fdbb152609c859169fcad85769147213f5223448268240862e4c94c3bc4cf9da"}
Jul 15 15:32:18.948011 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.947986    2580 generic.go:356] "Generic (PLEG): container finished" podID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerID="bcf0df6dd4e05a014ab98e343f9f3b4985a14178bf4c2051d305a7d55f1ea53c" exitCode=0
Jul 15 15:32:18.948094 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:18.948020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z" event={"ID":"3fe1f4e4-3a2d-4173-9714-e239fb122f32","Type":"ContainerDied","Data":"bcf0df6dd4e05a014ab98e343f9f3b4985a14178bf4c2051d305a7d55f1ea53c"}
Jul 15 15:32:19.961573 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.961539    2580 generic.go:356] "Generic (PLEG): container finished" podID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerID="ab362d50508a83873a57951b9dc26d74059b946c8d90b6cb98031b735a14318d" exitCode=0
Jul 15 15:32:19.962018 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.961623    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z" event={"ID":"3fe1f4e4-3a2d-4173-9714-e239fb122f32","Type":"ContainerDied","Data":"ab362d50508a83873a57951b9dc26d74059b946c8d90b6cb98031b735a14318d"}
Jul 15 15:32:19.963454 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.963433    2580 generic.go:356] "Generic (PLEG): container finished" podID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerID="49ecea9bc8b28a457ceb0187f7c225ae9aeb66bfb45b793a22b8cdfead0df5d7" exitCode=0
Jul 15 15:32:19.963554 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.963498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v" event={"ID":"a42fccbc-1d50-4ae0-91a2-763b44861dd9","Type":"ContainerDied","Data":"49ecea9bc8b28a457ceb0187f7c225ae9aeb66bfb45b793a22b8cdfead0df5d7"}
Jul 15 15:32:19.965453 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.965429    2580 generic.go:356] "Generic (PLEG): container finished" podID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerID="678a70d52014ed33ae8b91df5617fa4a7dd24f6d872b84b9380dc21f510cb78f" exitCode=0
Jul 15 15:32:19.965543 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.965511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4" event={"ID":"b46c748a-0c0e-45fb-8a85-0ba04cb78361","Type":"ContainerDied","Data":"678a70d52014ed33ae8b91df5617fa4a7dd24f6d872b84b9380dc21f510cb78f"}
Jul 15 15:32:19.967723 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.967703    2580 generic.go:356] "Generic (PLEG): container finished" podID="d158794c-7d7d-42c8-8364-72aac59b575b" containerID="80b641dc530a32cd855efbf9bdbb400923785f74a85f184cc3f9acb77b64eac7" exitCode=0
Jul 15 15:32:19.967810 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:19.967776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc" event={"ID":"d158794c-7d7d-42c8-8364-72aac59b575b","Type":"ContainerDied","Data":"80b641dc530a32cd855efbf9bdbb400923785f74a85f184cc3f9acb77b64eac7"}
Jul 15 15:32:20.975330 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:20.975248    2580 generic.go:356] "Generic (PLEG): container finished" podID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerID="a8dfb6de81e86fe15cb85a9dbeca5ab479827fdb10c41bc6296a4310da9f5a88" exitCode=0
Jul 15 15:32:20.975773 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:20.975332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v" event={"ID":"a42fccbc-1d50-4ae0-91a2-763b44861dd9","Type":"ContainerDied","Data":"a8dfb6de81e86fe15cb85a9dbeca5ab479827fdb10c41bc6296a4310da9f5a88"}
Jul 15 15:32:20.977429 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:20.977406    2580 generic.go:356] "Generic (PLEG): container finished" podID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerID="2eca5d26dee2e22fd2cef2cd698c1e281e564cdd0ae62f64a52852a15d44e17a" exitCode=0
Jul 15 15:32:20.977534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:20.977432    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4" event={"ID":"b46c748a-0c0e-45fb-8a85-0ba04cb78361","Type":"ContainerDied","Data":"2eca5d26dee2e22fd2cef2cd698c1e281e564cdd0ae62f64a52852a15d44e17a"}
Jul 15 15:32:21.103844 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.103820    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:21.142484 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.142461    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:21.183825 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.183800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-bundle\") pod \"d158794c-7d7d-42c8-8364-72aac59b575b\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") "
Jul 15 15:32:21.183951 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.183834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-bundle\") pod \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") "
Jul 15 15:32:21.183951 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.183856    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z864t\" (UniqueName: \"kubernetes.io/projected/3fe1f4e4-3a2d-4173-9714-e239fb122f32-kube-api-access-z864t\") pod \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") "
Jul 15 15:32:21.183951 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.183872    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-util\") pod \"d158794c-7d7d-42c8-8364-72aac59b575b\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") "
Jul 15 15:32:21.183951 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.183907    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-util\") pod \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\" (UID: \"3fe1f4e4-3a2d-4173-9714-e239fb122f32\") "
Jul 15 15:32:21.184173 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.183996    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg25j\" (UniqueName: \"kubernetes.io/projected/d158794c-7d7d-42c8-8364-72aac59b575b-kube-api-access-cg25j\") pod \"d158794c-7d7d-42c8-8364-72aac59b575b\" (UID: \"d158794c-7d7d-42c8-8364-72aac59b575b\") "
Jul 15 15:32:21.184436 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.184412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-bundle" (OuterVolumeSpecName: "bundle") pod "d158794c-7d7d-42c8-8364-72aac59b575b" (UID: "d158794c-7d7d-42c8-8364-72aac59b575b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:21.184533 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.184501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-bundle" (OuterVolumeSpecName: "bundle") pod "3fe1f4e4-3a2d-4173-9714-e239fb122f32" (UID: "3fe1f4e4-3a2d-4173-9714-e239fb122f32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:21.186061 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.186034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fe1f4e4-3a2d-4173-9714-e239fb122f32-kube-api-access-z864t" (OuterVolumeSpecName: "kube-api-access-z864t") pod "3fe1f4e4-3a2d-4173-9714-e239fb122f32" (UID: "3fe1f4e4-3a2d-4173-9714-e239fb122f32"). InnerVolumeSpecName "kube-api-access-z864t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:32:21.186480 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.186454    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d158794c-7d7d-42c8-8364-72aac59b575b-kube-api-access-cg25j" (OuterVolumeSpecName: "kube-api-access-cg25j") pod "d158794c-7d7d-42c8-8364-72aac59b575b" (UID: "d158794c-7d7d-42c8-8364-72aac59b575b"). InnerVolumeSpecName "kube-api-access-cg25j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:32:21.188964 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.188933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-util" (OuterVolumeSpecName: "util") pod "d158794c-7d7d-42c8-8364-72aac59b575b" (UID: "d158794c-7d7d-42c8-8364-72aac59b575b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:21.190401 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.190377    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-util" (OuterVolumeSpecName: "util") pod "3fe1f4e4-3a2d-4173-9714-e239fb122f32" (UID: "3fe1f4e4-3a2d-4173-9714-e239fb122f32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:21.285003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.284932    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg25j\" (UniqueName: \"kubernetes.io/projected/d158794c-7d7d-42c8-8364-72aac59b575b-kube-api-access-cg25j\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:21.285003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.284959    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:21.285003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.284970    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:21.285003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.284978    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z864t\" (UniqueName: \"kubernetes.io/projected/3fe1f4e4-3a2d-4173-9714-e239fb122f32-kube-api-access-z864t\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:21.285003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.284987    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d158794c-7d7d-42c8-8364-72aac59b575b-util\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:21.285003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.284995    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fe1f4e4-3a2d-4173-9714-e239fb122f32-util\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:21.984797 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.984688    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc"
Jul 15 15:32:21.984797 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.984703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc" event={"ID":"d158794c-7d7d-42c8-8364-72aac59b575b","Type":"ContainerDied","Data":"07f6acc197f36505e32dfdac9b12df09eb0c931d6750ff9bcbb4abe6589d43b5"}
Jul 15 15:32:21.984797 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.984765    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07f6acc197f36505e32dfdac9b12df09eb0c931d6750ff9bcbb4abe6589d43b5"
Jul 15 15:32:21.986889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.986791    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z"
Jul 15 15:32:21.986889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.986792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z" event={"ID":"3fe1f4e4-3a2d-4173-9714-e239fb122f32","Type":"ContainerDied","Data":"24b41da3d1459da87eee6e3a5ce96757fa6efdc8f7bd1cfd17d38a814f7b83fe"}
Jul 15 15:32:21.986889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:21.986861    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24b41da3d1459da87eee6e3a5ce96757fa6efdc8f7bd1cfd17d38a814f7b83fe"
Jul 15 15:32:22.121237 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.121214    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:22.145943 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.145923    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:22.191504 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.191475    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-util\") pod \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") "
Jul 15 15:32:22.191624 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.191508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq675\" (UniqueName: \"kubernetes.io/projected/b46c748a-0c0e-45fb-8a85-0ba04cb78361-kube-api-access-hq675\") pod \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") "
Jul 15 15:32:22.191624 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.191546    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-bundle\") pod \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") "
Jul 15 15:32:22.191726 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.191655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-util\") pod \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") "
Jul 15 15:32:22.191726 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.191695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-bundle\") pod \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\" (UID: \"b46c748a-0c0e-45fb-8a85-0ba04cb78361\") "
Jul 15 15:32:22.191871 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.191787    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw4b6\" (UniqueName: \"kubernetes.io/projected/a42fccbc-1d50-4ae0-91a2-763b44861dd9-kube-api-access-zw4b6\") pod \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\" (UID: \"a42fccbc-1d50-4ae0-91a2-763b44861dd9\") "
Jul 15 15:32:22.192179 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.192149    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-bundle" (OuterVolumeSpecName: "bundle") pod "a42fccbc-1d50-4ae0-91a2-763b44861dd9" (UID: "a42fccbc-1d50-4ae0-91a2-763b44861dd9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:22.192434 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.192402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-bundle" (OuterVolumeSpecName: "bundle") pod "b46c748a-0c0e-45fb-8a85-0ba04cb78361" (UID: "b46c748a-0c0e-45fb-8a85-0ba04cb78361"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:22.193644 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.193615    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46c748a-0c0e-45fb-8a85-0ba04cb78361-kube-api-access-hq675" (OuterVolumeSpecName: "kube-api-access-hq675") pod "b46c748a-0c0e-45fb-8a85-0ba04cb78361" (UID: "b46c748a-0c0e-45fb-8a85-0ba04cb78361"). InnerVolumeSpecName "kube-api-access-hq675". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:32:22.194149 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.194130    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a42fccbc-1d50-4ae0-91a2-763b44861dd9-kube-api-access-zw4b6" (OuterVolumeSpecName: "kube-api-access-zw4b6") pod "a42fccbc-1d50-4ae0-91a2-763b44861dd9" (UID: "a42fccbc-1d50-4ae0-91a2-763b44861dd9"). InnerVolumeSpecName "kube-api-access-zw4b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:32:22.197598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.197566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-util" (OuterVolumeSpecName: "util") pod "b46c748a-0c0e-45fb-8a85-0ba04cb78361" (UID: "b46c748a-0c0e-45fb-8a85-0ba04cb78361"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:22.197912 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.197892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-util" (OuterVolumeSpecName: "util") pod "a42fccbc-1d50-4ae0-91a2-763b44861dd9" (UID: "a42fccbc-1d50-4ae0-91a2-763b44861dd9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:32:22.293079 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.293022    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-util\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:22.293079 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.293044    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq675\" (UniqueName: \"kubernetes.io/projected/b46c748a-0c0e-45fb-8a85-0ba04cb78361-kube-api-access-hq675\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:22.293079 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.293053    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:22.293079 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.293062    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a42fccbc-1d50-4ae0-91a2-763b44861dd9-util\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:22.293079 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.293070    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b46c748a-0c0e-45fb-8a85-0ba04cb78361-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:22.293079 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.293077    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw4b6\" (UniqueName: \"kubernetes.io/projected/a42fccbc-1d50-4ae0-91a2-763b44861dd9-kube-api-access-zw4b6\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:22.993613 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.993530    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v"
Jul 15 15:32:22.994050 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.993526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v" event={"ID":"a42fccbc-1d50-4ae0-91a2-763b44861dd9","Type":"ContainerDied","Data":"de62ddb6a6e23e972c90c22384750d8198b71402454d86d866ee528f6758e64d"}
Jul 15 15:32:22.994050 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.993671    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de62ddb6a6e23e972c90c22384750d8198b71402454d86d866ee528f6758e64d"
Jul 15 15:32:22.995446 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.995423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4" event={"ID":"b46c748a-0c0e-45fb-8a85-0ba04cb78361","Type":"ContainerDied","Data":"ba2104fb3b1b6832ce03f037b22fa80c44546b6800fab68b9d174de98d903f39"}
Jul 15 15:32:22.995552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.995458    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba2104fb3b1b6832ce03f037b22fa80c44546b6800fab68b9d174de98d903f39"
Jul 15 15:32:22.995552 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:22.995429    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4"
Jul 15 15:32:23.396947 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.396917    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9c55965-gwdg8"]
Jul 15 15:32:23.397276 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397263    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerName="pull"
Jul 15 15:32:23.397325 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397278    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerName="pull"
Jul 15 15:32:23.397325 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397287    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerName="util"
Jul 15 15:32:23.397325 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397293    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerName="util"
Jul 15 15:32:23.397325 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397302    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerName="extract"
Jul 15 15:32:23.397325 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397307    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerName="extract"
Jul 15 15:32:23.397325 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397317    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerName="util"
Jul 15 15:32:23.397325 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397322    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerName="util"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397335    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerName="util"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397340    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerName="util"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397346    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d158794c-7d7d-42c8-8364-72aac59b575b" containerName="pull"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397351    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d158794c-7d7d-42c8-8364-72aac59b575b" containerName="pull"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397358    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d158794c-7d7d-42c8-8364-72aac59b575b" containerName="util"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397363    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d158794c-7d7d-42c8-8364-72aac59b575b" containerName="util"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397371    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397377    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397386    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397390    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397400    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerName="pull"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397406    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerName="pull"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397413    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerName="pull"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397418    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerName="pull"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397424    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d158794c-7d7d-42c8-8364-72aac59b575b" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397429    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d158794c-7d7d-42c8-8364-72aac59b575b" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397478    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe1f4e4-3a2d-4173-9714-e239fb122f32" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397488    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a42fccbc-1d50-4ae0-91a2-763b44861dd9" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397494    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b46c748a-0c0e-45fb-8a85-0ba04cb78361" containerName="extract"
Jul 15 15:32:23.397618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.397502    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d158794c-7d7d-42c8-8364-72aac59b575b" containerName="extract"
Jul 15 15:32:23.401509 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.401485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.410463 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.410441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9c55965-gwdg8"]
Jul 15 15:32:23.504363 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.504333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-config\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.504363 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.504365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-trusted-ca-bundle\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.504527 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.504389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-service-ca\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.504527 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.504483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-serving-cert\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.504527 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.504508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-oauth-config\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.504622 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.504588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hkvs\" (UniqueName: \"kubernetes.io/projected/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-kube-api-access-2hkvs\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.504656 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.504619    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-oauth-serving-cert\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.605734 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.605699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hkvs\" (UniqueName: \"kubernetes.io/projected/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-kube-api-access-2hkvs\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.605888 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.605741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-oauth-serving-cert\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.605888 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.605798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-config\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.605888 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.605815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-trusted-ca-bundle\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.605888 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.605842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-service-ca\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.606114 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.605915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-serving-cert\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.606114 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.605938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-oauth-config\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.606520 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.606489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-oauth-serving-cert\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.606689 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.606661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-config\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.606812 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.606721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-trusted-ca-bundle\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.606896 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.606872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-service-ca\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.608430 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.608407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-oauth-config\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.608505 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.608430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-console-serving-cert\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.618603 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.618580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hkvs\" (UniqueName: \"kubernetes.io/projected/678240ac-3a0f-4287-92d3-fa1b5b6a79bc-kube-api-access-2hkvs\") pod \"console-c9c55965-gwdg8\" (UID: \"678240ac-3a0f-4287-92d3-fa1b5b6a79bc\") " pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.711448 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.711380    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:23.845038 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:23.845014    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9c55965-gwdg8"]
Jul 15 15:32:23.846830 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:32:23.846804    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678240ac_3a0f_4287_92d3_fa1b5b6a79bc.slice/crio-3735cb2bf755bcf9ee7f0f892c37fb3514133a769ae36bb924c5a3b9c1b4d63a WatchSource:0}: Error finding container 3735cb2bf755bcf9ee7f0f892c37fb3514133a769ae36bb924c5a3b9c1b4d63a: Status 404 returned error can't find the container with id 3735cb2bf755bcf9ee7f0f892c37fb3514133a769ae36bb924c5a3b9c1b4d63a
Jul 15 15:32:24.003058 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:24.002975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9c55965-gwdg8" event={"ID":"678240ac-3a0f-4287-92d3-fa1b5b6a79bc","Type":"ContainerStarted","Data":"2331acb25c7fa5d44f0dc85428a288af9e07e63b2b319fe507d1c1aac45ed325"}
Jul 15 15:32:24.003058 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:24.003031    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9c55965-gwdg8" event={"ID":"678240ac-3a0f-4287-92d3-fa1b5b6a79bc","Type":"ContainerStarted","Data":"3735cb2bf755bcf9ee7f0f892c37fb3514133a769ae36bb924c5a3b9c1b4d63a"}
Jul 15 15:32:24.023546 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:24.023498    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9c55965-gwdg8" podStartSLOduration=1.0234821649999999 podStartE2EDuration="1.023482165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:32:24.022407689 +0000 UTC m=+468.823867503" watchObservedRunningTime="2026-07-15 15:32:24.023482165 +0000 UTC m=+468.824941991"
Jul 15 15:32:33.711839 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:33.711800    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:33.712281 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:33.711872    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:33.716288 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:33.716266    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:34.057549 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:34.057471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9c55965-gwdg8"
Jul 15 15:32:34.112256 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:34.112222    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bcdc5ccc-dk78x"]
Jul 15 15:32:44.519522 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.519484    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"]
Jul 15 15:32:44.527727 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.527701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"
Jul 15 15:32:44.531945 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.531911    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 15:32:44.533229 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.533210    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 15:32:44.533331 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.533314    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 15:32:44.533331 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.533319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-gdlcp\""
Jul 15 15:32:44.535287 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.535265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"]
Jul 15 15:32:44.589888 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.589854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6842\" (UniqueName: \"kubernetes.io/projected/b755299d-7f20-496c-bd1b-93a92d33c8b5-kube-api-access-v6842\") pod \"dns-operator-controller-manager-864dd78c4c-dz7wx\" (UID: \"b755299d-7f20-496c-bd1b-93a92d33c8b5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"
Jul 15 15:32:44.690663 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.690633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6842\" (UniqueName: \"kubernetes.io/projected/b755299d-7f20-496c-bd1b-93a92d33c8b5-kube-api-access-v6842\") pod \"dns-operator-controller-manager-864dd78c4c-dz7wx\" (UID: \"b755299d-7f20-496c-bd1b-93a92d33c8b5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"
Jul 15 15:32:44.710458 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.710435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6842\" (UniqueName: \"kubernetes.io/projected/b755299d-7f20-496c-bd1b-93a92d33c8b5-kube-api-access-v6842\") pod \"dns-operator-controller-manager-864dd78c4c-dz7wx\" (UID: \"b755299d-7f20-496c-bd1b-93a92d33c8b5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"
Jul 15 15:32:44.839368 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.839343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"
Jul 15 15:32:44.981720 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:44.981649    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"]
Jul 15 15:32:44.984013 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:32:44.983988    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb755299d_7f20_496c_bd1b_93a92d33c8b5.slice/crio-06678509b3b822cdb9a037d76a56166ec781dd9da3f6d1fe549dce61d78ef048 WatchSource:0}: Error finding container 06678509b3b822cdb9a037d76a56166ec781dd9da3f6d1fe549dce61d78ef048: Status 404 returned error can't find the container with id 06678509b3b822cdb9a037d76a56166ec781dd9da3f6d1fe549dce61d78ef048
Jul 15 15:32:45.113677 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:45.113609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx" event={"ID":"b755299d-7f20-496c-bd1b-93a92d33c8b5","Type":"ContainerStarted","Data":"06678509b3b822cdb9a037d76a56166ec781dd9da3f6d1fe549dce61d78ef048"}
Jul 15 15:32:49.137332 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:49.137298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx" event={"ID":"b755299d-7f20-496c-bd1b-93a92d33c8b5","Type":"ContainerStarted","Data":"cef5442e128e540324ad325f45a0bc28c7e6945a7e2ebb78140b7e61acfd6945"}
Jul 15 15:32:49.137745 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:49.137389    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"
Jul 15 15:32:49.172578 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:49.172532    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx" podStartSLOduration=1.8773221759999998 podStartE2EDuration="5.172515576s" totalImagesPullingTime="3.2951934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:32:49.167769157 +0000 UTC m=+493.969228961" watchObservedRunningTime="2026-07-15 15:32:49.172515576 +0000 UTC m=+493.973975402"
Jul 15 15:32:59.132981 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.132932    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8bcdc5ccc-dk78x" podUID="f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" containerName="console" containerID="cri-o://9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3" gracePeriod=15
Jul 15 15:32:59.376459 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.376435    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:32:59.521944 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.521862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-serving-cert\") pod \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") "
Jul 15 15:32:59.521944 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.521921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-trusted-ca-bundle\") pod \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") "
Jul 15 15:32:59.522150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.521973    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-oauth-serving-cert\") pod \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") "
Jul 15 15:32:59.522150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-oauth-config\") pod \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") "
Jul 15 15:32:59.522150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522121    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-service-ca\") pod \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") "
Jul 15 15:32:59.522306 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522183    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptth8\" (UniqueName: \"kubernetes.io/projected/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-kube-api-access-ptth8\") pod \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") "
Jul 15 15:32:59.522306 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522227    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-config\") pod \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\" (UID: \"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b\") "
Jul 15 15:32:59.522458 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522414    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" (UID: "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:32:59.522458 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522424    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" (UID: "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:32:59.522612 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522463    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-service-ca" (OuterVolumeSpecName: "service-ca") pod "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" (UID: "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:32:59.522612 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522567    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-trusted-ca-bundle\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:59.522612 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522587    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-oauth-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:59.522784 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.522652    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-config" (OuterVolumeSpecName: "console-config") pod "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" (UID: "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:32:59.524080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.524054    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" (UID: "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:32:59.524214 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.524194    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" (UID: "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:32:59.524484 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.524462    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-kube-api-access-ptth8" (OuterVolumeSpecName: "kube-api-access-ptth8") pod "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" (UID: "f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b"). InnerVolumeSpecName "kube-api-access-ptth8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:32:59.624090 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.624054    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-oauth-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:59.624090 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.624086    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:59.624277 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.624098    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptth8\" (UniqueName: \"kubernetes.io/projected/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-kube-api-access-ptth8\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:59.624277 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.624110    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-config\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:32:59.624277 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:32:59.624122    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b-console-serving-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:33:00.145294 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.145267    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dz7wx"
Jul 15 15:33:00.197789 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.197738    2580 generic.go:356] "Generic (PLEG): container finished" podID="f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" containerID="9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3" exitCode=0
Jul 15 15:33:00.197963 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.197811    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcdc5ccc-dk78x"
Jul 15 15:33:00.197963 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.197828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcdc5ccc-dk78x" event={"ID":"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b","Type":"ContainerDied","Data":"9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3"}
Jul 15 15:33:00.197963 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.197899    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcdc5ccc-dk78x" event={"ID":"f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b","Type":"ContainerDied","Data":"399f47381fd440fe75a5b180343b0403345fba6f43efa8e7df360fa42314ca6c"}
Jul 15 15:33:00.197963 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.197928    2580 scope.go:117] "RemoveContainer" containerID="9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3"
Jul 15 15:33:00.208329 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.208278    2580 scope.go:117] "RemoveContainer" containerID="9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3"
Jul 15 15:33:00.208569 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:33:00.208551    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3\": container with ID starting with 9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3 not found: ID does not exist" containerID="9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3"
Jul 15 15:33:00.208618 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.208575    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3"} err="failed to get container status \"9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3\": rpc error: code = NotFound desc = could not find container \"9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3\": container with ID starting with 9a95bac431a51fb7ab15f0ec0354b32924d69fd08b114136e85a767404f56de3 not found: ID does not exist"
Jul 15 15:33:00.225006 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.224982    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bcdc5ccc-dk78x"]
Jul 15 15:33:00.230411 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.230392    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8bcdc5ccc-dk78x"]
Jul 15 15:33:00.783138 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.783099    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"]
Jul 15 15:33:00.784027 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.784003    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" containerName="console"
Jul 15 15:33:00.784027 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.784027    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" containerName="console"
Jul 15 15:33:00.784232 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.784219    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" containerName="console"
Jul 15 15:33:00.789436 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.789414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:00.792246 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.792226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nsmdv\""
Jul 15 15:33:00.802132 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.802109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"]
Jul 15 15:33:00.933129 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.933091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq7dv\" (UniqueName: \"kubernetes.io/projected/e7f0936b-301a-4848-ad60-e0e0aaf0714c-kube-api-access-pq7dv\") pod \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" (UID: \"e7f0936b-301a-4848-ad60-e0e0aaf0714c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:00.933302 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.933142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7f0936b-301a-4848-ad60-e0e0aaf0714c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" (UID: \"e7f0936b-301a-4848-ad60-e0e0aaf0714c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:00.954255 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.954217    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"]
Jul 15 15:33:00.954519 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:33:00.954499    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-pq7dv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4" podUID="e7f0936b-301a-4848-ad60-e0e0aaf0714c"
Jul 15 15:33:00.964940 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.964911    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"]
Jul 15 15:33:00.989913 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.989883    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"]
Jul 15 15:33:00.993573 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:00.993546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.020972 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.020941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"]
Jul 15 15:33:01.034298 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.034233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7f0936b-301a-4848-ad60-e0e0aaf0714c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" (UID: \"e7f0936b-301a-4848-ad60-e0e0aaf0714c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:01.034409 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.034343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7dv\" (UniqueName: \"kubernetes.io/projected/e7f0936b-301a-4848-ad60-e0e0aaf0714c-kube-api-access-pq7dv\") pod \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" (UID: \"e7f0936b-301a-4848-ad60-e0e0aaf0714c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:01.034687 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.034661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7f0936b-301a-4848-ad60-e0e0aaf0714c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" (UID: \"e7f0936b-301a-4848-ad60-e0e0aaf0714c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:01.040901 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:33:01.040879    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pq7dv for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-bhcb4" not found
Jul 15 15:33:01.041030 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:33:01.040942    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0936b-301a-4848-ad60-e0e0aaf0714c-kube-api-access-pq7dv podName:e7f0936b-301a-4848-ad60-e0e0aaf0714c nodeName:}" failed. No retries permitted until 2026-07-15 15:33:01.540926402 +0000 UTC m=+506.342386214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pq7dv" (UniqueName: "kubernetes.io/projected/e7f0936b-301a-4848-ad60-e0e0aaf0714c-kube-api-access-pq7dv") pod "kuadrant-operator-controller-manager-86878c769f-bhcb4" (UID: "e7f0936b-301a-4848-ad60-e0e0aaf0714c") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-bhcb4" not found
Jul 15 15:33:01.134826 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.134796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz6cr\" (UniqueName: \"kubernetes.io/projected/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-kube-api-access-bz6cr\") pod \"kuadrant-operator-controller-manager-86878c769f-796q7\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.134951 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.134910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-796q7\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.204473 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.204452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:01.209117 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:33:01.209091    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object" podUID="e7f0936b-301a-4848-ad60-e0e0aaf0714c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:01.211127 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.211110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:01.214030 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:33:01.214008    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object" podUID="e7f0936b-301a-4848-ad60-e0e0aaf0714c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:01.235424 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.235402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-796q7\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.235502 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.235460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz6cr\" (UniqueName: \"kubernetes.io/projected/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-kube-api-access-bz6cr\") pod \"kuadrant-operator-controller-manager-86878c769f-796q7\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.235735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.235719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-796q7\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.248107 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.248087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz6cr\" (UniqueName: \"kubernetes.io/projected/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-kube-api-access-bz6cr\") pod \"kuadrant-operator-controller-manager-86878c769f-796q7\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.304064 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.304009    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:01.336740 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.336719    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7f0936b-301a-4848-ad60-e0e0aaf0714c-extensions-socket-volume\") pod \"e7f0936b-301a-4848-ad60-e0e0aaf0714c\" (UID: \"e7f0936b-301a-4848-ad60-e0e0aaf0714c\") "
Jul 15 15:33:01.336992 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.336970    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f0936b-301a-4848-ad60-e0e0aaf0714c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7f0936b-301a-4848-ad60-e0e0aaf0714c" (UID: "e7f0936b-301a-4848-ad60-e0e0aaf0714c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:33:01.337073 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.337062    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq7dv\" (UniqueName: \"kubernetes.io/projected/e7f0936b-301a-4848-ad60-e0e0aaf0714c-kube-api-access-pq7dv\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:33:01.337109 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.337074    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7f0936b-301a-4848-ad60-e0e0aaf0714c-extensions-socket-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:33:01.457708 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.457684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"]
Jul 15 15:33:01.459741 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:33:01.459710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52e99d5_b3a7_4daa_a2a5_dc3e8a077852.slice/crio-9d400018e5c701a167cfc8f09d931004917dee98c2ad28434fff1bb9db9a1c00 WatchSource:0}: Error finding container 9d400018e5c701a167cfc8f09d931004917dee98c2ad28434fff1bb9db9a1c00: Status 404 returned error can't find the container with id 9d400018e5c701a167cfc8f09d931004917dee98c2ad28434fff1bb9db9a1c00
Jul 15 15:33:01.767257 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.767226    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7f0936b-301a-4848-ad60-e0e0aaf0714c" path="/var/lib/kubelet/pods/e7f0936b-301a-4848-ad60-e0e0aaf0714c/volumes"
Jul 15 15:33:01.767526 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:01.767513    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b" path="/var/lib/kubelet/pods/f5ca15b2-7aec-42a4-a2cb-a7c28fb3cb5b/volumes"
Jul 15 15:33:02.212985 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:02.212950    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:02.212985 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:02.212966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7" event={"ID":"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852","Type":"ContainerStarted","Data":"9d400018e5c701a167cfc8f09d931004917dee98c2ad28434fff1bb9db9a1c00"}
Jul 15 15:33:02.218280 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:33:02.218246    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-bhcb4\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object" podUID="e7f0936b-301a-4848-ad60-e0e0aaf0714c" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bhcb4"
Jul 15 15:33:07.243769 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:07.243715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7" event={"ID":"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852","Type":"ContainerStarted","Data":"cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44"}
Jul 15 15:33:07.244146 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:07.243853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:07.269856 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:07.269809    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7" podStartSLOduration=2.401097275 podStartE2EDuration="7.269794887s" totalImagesPullingTime="4.868697612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:33:07.266233695 +0000 UTC m=+512.067693509" watchObservedRunningTime="2026-07-15 15:33:07.269794887 +0000 UTC m=+512.071254700"
Jul 15 15:33:18.250969 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:18.250936    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:33:41.768041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.767995    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:33:41.772345 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.772328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:41.774950 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.774931    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7n4mz\""
Jul 15 15:33:41.775060 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.774935    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 15:33:41.781184 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.781160    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:33:41.814521 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.814491    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:33:41.904409 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.904383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e35a104-6234-4a2b-9402-3472949cda4e-config-file\") pod \"limitador-limitador-54d6d988d5-6hw2j\" (UID: \"3e35a104-6234-4a2b-9402-3472949cda4e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:41.904548 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:41.904414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfm2v\" (UniqueName: \"kubernetes.io/projected/3e35a104-6234-4a2b-9402-3472949cda4e-kube-api-access-pfm2v\") pod \"limitador-limitador-54d6d988d5-6hw2j\" (UID: \"3e35a104-6234-4a2b-9402-3472949cda4e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:42.005026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:42.004989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e35a104-6234-4a2b-9402-3472949cda4e-config-file\") pod \"limitador-limitador-54d6d988d5-6hw2j\" (UID: \"3e35a104-6234-4a2b-9402-3472949cda4e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:42.005199 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:42.005033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfm2v\" (UniqueName: \"kubernetes.io/projected/3e35a104-6234-4a2b-9402-3472949cda4e-kube-api-access-pfm2v\") pod \"limitador-limitador-54d6d988d5-6hw2j\" (UID: \"3e35a104-6234-4a2b-9402-3472949cda4e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:42.005634 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:42.005613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e35a104-6234-4a2b-9402-3472949cda4e-config-file\") pod \"limitador-limitador-54d6d988d5-6hw2j\" (UID: \"3e35a104-6234-4a2b-9402-3472949cda4e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:42.013927 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:42.013901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfm2v\" (UniqueName: \"kubernetes.io/projected/3e35a104-6234-4a2b-9402-3472949cda4e-kube-api-access-pfm2v\") pod \"limitador-limitador-54d6d988d5-6hw2j\" (UID: \"3e35a104-6234-4a2b-9402-3472949cda4e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:42.083181 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:42.083155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:42.220521 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:42.220499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:33:42.223403 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:33:42.223377    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e35a104_6234_4a2b_9402_3472949cda4e.slice/crio-6c318def0c5bfbe6881e5d5a0a52d2a9745e84a96e7e86c0b1137fd4ba216aff WatchSource:0}: Error finding container 6c318def0c5bfbe6881e5d5a0a52d2a9745e84a96e7e86c0b1137fd4ba216aff: Status 404 returned error can't find the container with id 6c318def0c5bfbe6881e5d5a0a52d2a9745e84a96e7e86c0b1137fd4ba216aff
Jul 15 15:33:42.425995 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:42.425914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j" event={"ID":"3e35a104-6234-4a2b-9402-3472949cda4e","Type":"ContainerStarted","Data":"6c318def0c5bfbe6881e5d5a0a52d2a9745e84a96e7e86c0b1137fd4ba216aff"}
Jul 15 15:33:45.443938 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:45.443901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j" event={"ID":"3e35a104-6234-4a2b-9402-3472949cda4e","Type":"ContainerStarted","Data":"1c71d7967105b297cda4ee920fc531792cf2adf6b917c858fade347c4e0d024c"}
Jul 15 15:33:45.444399 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:45.443982    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:33:45.463230 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:45.463184    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j" podStartSLOduration=1.9048854400000002 podStartE2EDuration="4.463169534s" totalImagesPullingTime="2.558284094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:33:45.460488446 +0000 UTC m=+550.261948275" watchObservedRunningTime="2026-07-15 15:33:45.463169534 +0000 UTC m=+550.264629358"
Jul 15 15:33:56.450955 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:33:56.450919    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"
Jul 15 15:34:11.423473 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.423437    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-664674b975-x9k48"]
Jul 15 15:34:11.427135 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.427111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.430626 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.430607    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dwnh6\""
Jul 15 15:34:11.430742 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.430609    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 15 15:34:11.434278 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.434209    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664674b975-x9k48"]
Jul 15 15:34:11.457512 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.457478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fp86\" (UniqueName: \"kubernetes.io/projected/0a7db30a-2837-463e-95eb-598acc28d1b7-kube-api-access-9fp86\") pod \"authorino-664674b975-x9k48\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") " pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.457661 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.457604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a7db30a-2837-463e-95eb-598acc28d1b7-tls-cert\") pod \"authorino-664674b975-x9k48\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") " pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.558146 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.558110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fp86\" (UniqueName: \"kubernetes.io/projected/0a7db30a-2837-463e-95eb-598acc28d1b7-kube-api-access-9fp86\") pod \"authorino-664674b975-x9k48\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") " pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.558320 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.558260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a7db30a-2837-463e-95eb-598acc28d1b7-tls-cert\") pod \"authorino-664674b975-x9k48\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") " pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.560696 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.560672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a7db30a-2837-463e-95eb-598acc28d1b7-tls-cert\") pod \"authorino-664674b975-x9k48\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") " pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.566182 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.566160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fp86\" (UniqueName: \"kubernetes.io/projected/0a7db30a-2837-463e-95eb-598acc28d1b7-kube-api-access-9fp86\") pod \"authorino-664674b975-x9k48\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") " pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.736995 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.736911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:34:11.882834 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:11.882809    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664674b975-x9k48"]
Jul 15 15:34:11.884671 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:34:11.884641    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a7db30a_2837_463e_95eb_598acc28d1b7.slice/crio-ff2aafa95dc159bc859282a1c8df6c6f0e43c6dd77c660129a8e0302ea744155 WatchSource:0}: Error finding container ff2aafa95dc159bc859282a1c8df6c6f0e43c6dd77c660129a8e0302ea744155: Status 404 returned error can't find the container with id ff2aafa95dc159bc859282a1c8df6c6f0e43c6dd77c660129a8e0302ea744155
Jul 15 15:34:12.595271 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:12.595235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664674b975-x9k48" event={"ID":"0a7db30a-2837-463e-95eb-598acc28d1b7","Type":"ContainerStarted","Data":"ff2aafa95dc159bc859282a1c8df6c6f0e43c6dd77c660129a8e0302ea744155"}
Jul 15 15:34:15.616012 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:15.615980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664674b975-x9k48" event={"ID":"0a7db30a-2837-463e-95eb-598acc28d1b7","Type":"ContainerStarted","Data":"357eea7423caac613b0fde96dd02cecfbc1d0fa241f395c13934419dee22d812"}
Jul 15 15:34:15.632847 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:15.632802    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-664674b975-x9k48" podStartSLOduration=1.200827983 podStartE2EDuration="4.632785479s" totalImagesPullingTime="3.431957496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:34:15.632289078 +0000 UTC m=+580.433748935" watchObservedRunningTime="2026-07-15 15:34:15.632785479 +0000 UTC m=+580.434245293"
Jul 15 15:34:35.644708 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:35.644677    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:34:35.645866 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:35.645845    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:34:57.709427 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:34:57.709344    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:35:23.903804 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:23.903743    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:35:32.562279 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:32.562247    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:35:36.168301 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:36.168259    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:35:39.966272 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:39.966239    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"]
Jul 15 15:35:39.969820 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:39.969802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:39.973817 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:39.973794    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 15:35:39.973817 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:39.973810    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 15:35:39.973972 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:39.973811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 15 15:35:39.973972 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:39.973794    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bwbrz\""
Jul 15 15:35:39.979884 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:39.979859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"]
Jul 15 15:35:40.054652 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.054621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d922bffe-518d-4aad-abf3-635a0eadfb32-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.054831 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.054684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.054831 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.054742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.054831 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.054793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.054831 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.054818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfm65\" (UniqueName: \"kubernetes.io/projected/d922bffe-518d-4aad-abf3-635a0eadfb32-kube-api-access-xfm65\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.054979 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.054888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.155513 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.155478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.155671 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.155531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.155671 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.155547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.155671 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.155570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfm65\" (UniqueName: \"kubernetes.io/projected/d922bffe-518d-4aad-abf3-635a0eadfb32-kube-api-access-xfm65\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.155671 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.155612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.155671 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.155652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d922bffe-518d-4aad-abf3-635a0eadfb32-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.155950 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.155929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.156041 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.156020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.156136 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.156118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.158544 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.158514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d922bffe-518d-4aad-abf3-635a0eadfb32-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.158647 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.158619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d922bffe-518d-4aad-abf3-635a0eadfb32-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.165387 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.165361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfm65\" (UniqueName: \"kubernetes.io/projected/d922bffe-518d-4aad-abf3-635a0eadfb32-kube-api-access-xfm65\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr\" (UID: \"d922bffe-518d-4aad-abf3-635a0eadfb32\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.280254 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.280178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:40.421954 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.421924    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"]
Jul 15 15:35:40.424003 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:35:40.423978    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd922bffe_518d_4aad_abf3_635a0eadfb32.slice/crio-c47cc054a8cfa57fcaf468dea82246a92ebefc49b2618a2f92149108a616c6cf WatchSource:0}: Error finding container c47cc054a8cfa57fcaf468dea82246a92ebefc49b2618a2f92149108a616c6cf: Status 404 returned error can't find the container with id c47cc054a8cfa57fcaf468dea82246a92ebefc49b2618a2f92149108a616c6cf
Jul 15 15:35:40.425740 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:40.425723    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 15:35:41.078628 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:41.078577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr" event={"ID":"d922bffe-518d-4aad-abf3-635a0eadfb32","Type":"ContainerStarted","Data":"c47cc054a8cfa57fcaf468dea82246a92ebefc49b2618a2f92149108a616c6cf"}
Jul 15 15:35:41.470318 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:41.470228    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:35:42.085683 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:42.085645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr" event={"ID":"d922bffe-518d-4aad-abf3-635a0eadfb32","Type":"ContainerStarted","Data":"d459535d3870e0a114ca1d66148fd2deff05d9874effdf1a8d81c430447fd9cc"}
Jul 15 15:35:42.107600 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:42.107546    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr" podStartSLOduration=1.7430704129999999 podStartE2EDuration="3.107525971s" totalImagesPullingTime="1.364455558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:35:42.106730259 +0000 UTC m=+666.908190092" watchObservedRunningTime="2026-07-15 15:35:42.107525971 +0000 UTC m=+666.908985786"
Jul 15 15:35:47.177838 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:47.177784    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:35:50.280563 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:50.280524    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:50.281011 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:50.280598    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:50.293236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:50.293205    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:51.156205 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:51.156171    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr"
Jul 15 15:35:58.113862 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.113825    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"]
Jul 15 15:35:58.120505 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.120486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.123631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.123608    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 15 15:35:58.132050 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.132030    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"]
Jul 15 15:35:58.222180 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.222144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.222366 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.222289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.222366 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.222328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tpl4\" (UniqueName: \"kubernetes.io/projected/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-kube-api-access-4tpl4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.222518 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.222492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.222575 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.222547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.222635 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.222610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.323544 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.323508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.323706 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.323550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.323706 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.323586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.323706 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.323686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.323916 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.323725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tpl4\" (UniqueName: \"kubernetes.io/projected/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-kube-api-access-4tpl4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.323916 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.323825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.324018 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.323968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.324062 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.324043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.324176 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.324151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.325921 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.325899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.326080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.326064    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.338057 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.338023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tpl4\" (UniqueName: \"kubernetes.io/projected/b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8-kube-api-access-4tpl4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc\" (UID: \"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.430023 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.429952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:35:58.586803 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:58.586733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"]
Jul 15 15:35:58.588937 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:35:58.588910    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb51ddc1c_b7bc_4b8e_b049_42e6fc18aad8.slice/crio-8eaa38434250da68788d20a6c635719b768b4890409996993b5eb66743dfe5de WatchSource:0}: Error finding container 8eaa38434250da68788d20a6c635719b768b4890409996993b5eb66743dfe5de: Status 404 returned error can't find the container with id 8eaa38434250da68788d20a6c635719b768b4890409996993b5eb66743dfe5de
Jul 15 15:35:59.189886 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:59.189852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc" event={"ID":"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8","Type":"ContainerStarted","Data":"efb675e10d6f97219a215386eb606190be83f1954c5f956061ab640e097fa66f"}
Jul 15 15:35:59.189886 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:59.189891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc" event={"ID":"b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8","Type":"ContainerStarted","Data":"8eaa38434250da68788d20a6c635719b768b4890409996993b5eb66743dfe5de"}
Jul 15 15:35:59.215589 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:35:59.215533    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc" podStartSLOduration=0.988951366 podStartE2EDuration="1.215516781s" totalImagesPullingTime="226.565415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:35:59.214786656 +0000 UTC m=+684.016246472" watchObservedRunningTime="2026-07-15 15:35:59.215516781 +0000 UTC m=+684.016976599"
Jul 15 15:36:00.792375 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:00.792342    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:36:08.430991 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:08.430955    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:36:08.431523 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:08.431041    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:36:08.443763 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:08.443718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:36:09.260583 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:09.260555    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc"
Jul 15 15:36:58.320040 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.319932    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-664674b975-x9k48"]
Jul 15 15:36:58.320665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.320146    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-664674b975-x9k48" podUID="0a7db30a-2837-463e-95eb-598acc28d1b7" containerName="authorino" containerID="cri-o://357eea7423caac613b0fde96dd02cecfbc1d0fa241f395c13934419dee22d812" gracePeriod=30
Jul 15 15:36:58.532295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.532256    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a7db30a-2837-463e-95eb-598acc28d1b7" containerID="357eea7423caac613b0fde96dd02cecfbc1d0fa241f395c13934419dee22d812" exitCode=0
Jul 15 15:36:58.532433 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.532324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664674b975-x9k48" event={"ID":"0a7db30a-2837-463e-95eb-598acc28d1b7","Type":"ContainerDied","Data":"357eea7423caac613b0fde96dd02cecfbc1d0fa241f395c13934419dee22d812"}
Jul 15 15:36:58.580859 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.580811    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:36:58.669831 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.669804    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fp86\" (UniqueName: \"kubernetes.io/projected/0a7db30a-2837-463e-95eb-598acc28d1b7-kube-api-access-9fp86\") pod \"0a7db30a-2837-463e-95eb-598acc28d1b7\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") "
Jul 15 15:36:58.669997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.669923    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a7db30a-2837-463e-95eb-598acc28d1b7-tls-cert\") pod \"0a7db30a-2837-463e-95eb-598acc28d1b7\" (UID: \"0a7db30a-2837-463e-95eb-598acc28d1b7\") "
Jul 15 15:36:58.672024 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.671993    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7db30a-2837-463e-95eb-598acc28d1b7-kube-api-access-9fp86" (OuterVolumeSpecName: "kube-api-access-9fp86") pod "0a7db30a-2837-463e-95eb-598acc28d1b7" (UID: "0a7db30a-2837-463e-95eb-598acc28d1b7"). InnerVolumeSpecName "kube-api-access-9fp86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:36:58.680236 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.680205    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a7db30a-2837-463e-95eb-598acc28d1b7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0a7db30a-2837-463e-95eb-598acc28d1b7" (UID: "0a7db30a-2837-463e-95eb-598acc28d1b7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:36:58.771147 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.771113    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a7db30a-2837-463e-95eb-598acc28d1b7-tls-cert\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:36:58.771147 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:58.771144    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fp86\" (UniqueName: \"kubernetes.io/projected/0a7db30a-2837-463e-95eb-598acc28d1b7-kube-api-access-9fp86\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:36:59.539853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:59.539820    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664674b975-x9k48"
Jul 15 15:36:59.540245 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:59.539871    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664674b975-x9k48" event={"ID":"0a7db30a-2837-463e-95eb-598acc28d1b7","Type":"ContainerDied","Data":"ff2aafa95dc159bc859282a1c8df6c6f0e43c6dd77c660129a8e0302ea744155"}
Jul 15 15:36:59.540245 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:59.539921    2580 scope.go:117] "RemoveContainer" containerID="357eea7423caac613b0fde96dd02cecfbc1d0fa241f395c13934419dee22d812"
Jul 15 15:36:59.604484 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:59.604452    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-664674b975-x9k48"]
Jul 15 15:36:59.610314 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:59.610279    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-664674b975-x9k48"]
Jul 15 15:36:59.768100 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:36:59.768068    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7db30a-2837-463e-95eb-598acc28d1b7" path="/var/lib/kubelet/pods/0a7db30a-2837-463e-95eb-598acc28d1b7/volumes"
Jul 15 15:37:38.368266 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:37:38.368231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:37:49.363806 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:37:49.363769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:37:58.170849 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:37:58.170774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:38:05.964782 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:38:05.964739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:38:15.176118 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:38:15.176078    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:38:23.067590 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:38:23.067555    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:38:47.471894 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:38:47.471861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:38:58.366026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:38:58.365988    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:39:32.969404 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:39:32.969366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:39:35.682144 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:39:35.682115    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:39:35.684474 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:39:35.684453    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:39:43.575965 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:39:43.575933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:39:54.563162 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:39:54.563127    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:40:02.715677 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:40:02.715635    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:40:13.567005 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:40:13.566965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:40:22.667993 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:40:22.667959    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:40:34.566627 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:40:34.566590    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:40:44.270253 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:40:44.270213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:41:06.962697 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:41:06.962621    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:41:19.661781 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:41:19.661733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:41:43.770182 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:41:43.770152    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:41:54.270464 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:41:54.270430    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:42:07.970983 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:42:07.970951    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:42:18.373294 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:42:18.373250    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:42:35.069237 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:42:35.069151    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:43:13.242246 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:13.242199    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"]
Jul 15 15:43:13.242744 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:13.242422    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7" podUID="a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" containerName="manager" containerID="cri-o://cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44" gracePeriod=10
Jul 15 15:43:15.797817 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:15.797793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:43:15.871468 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:15.871386    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz6cr\" (UniqueName: \"kubernetes.io/projected/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-kube-api-access-bz6cr\") pod \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") "
Jul 15 15:43:15.871468 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:15.871454    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-extensions-socket-volume\") pod \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\" (UID: \"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852\") "
Jul 15 15:43:15.871872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:15.871846    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" (UID: "a52e99d5-b3a7-4daa-a2a5-dc3e8a077852"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:43:15.873437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:15.873413    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-kube-api-access-bz6cr" (OuterVolumeSpecName: "kube-api-access-bz6cr") pod "a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" (UID: "a52e99d5-b3a7-4daa-a2a5-dc3e8a077852"). InnerVolumeSpecName "kube-api-access-bz6cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:43:15.972147 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:15.972121    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-extensions-socket-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:43:15.972147 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:15.972142    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz6cr\" (UniqueName: \"kubernetes.io/projected/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852-kube-api-access-bz6cr\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:43:16.615402 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.615368    2580 generic.go:356] "Generic (PLEG): container finished" podID="a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" containerID="cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44" exitCode=0
Jul 15 15:43:16.615576 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.615414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"
Jul 15 15:43:16.615576 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.615416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7" event={"ID":"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852","Type":"ContainerDied","Data":"cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44"}
Jul 15 15:43:16.615576 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.615546    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7" event={"ID":"a52e99d5-b3a7-4daa-a2a5-dc3e8a077852","Type":"ContainerDied","Data":"9d400018e5c701a167cfc8f09d931004917dee98c2ad28434fff1bb9db9a1c00"}
Jul 15 15:43:16.615698 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.615579    2580 scope.go:117] "RemoveContainer" containerID="cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44"
Jul 15 15:43:16.626232 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.626210    2580 scope.go:117] "RemoveContainer" containerID="cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44"
Jul 15 15:43:16.626464 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:43:16.626447    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44\": container with ID starting with cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44 not found: ID does not exist" containerID="cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44"
Jul 15 15:43:16.626521 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.626471    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44"} err="failed to get container status \"cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44\": rpc error: code = NotFound desc = could not find container \"cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44\": container with ID starting with cb1000936b810239f800384497ba85116d600bc14699e8250ec8621bbf52ad44 not found: ID does not exist"
Jul 15 15:43:16.638622 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.638600    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"]
Jul 15 15:43:16.642144 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:16.642125    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-796q7"]
Jul 15 15:43:17.766985 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:43:17.766955    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" path="/var/lib/kubelet/pods/a52e99d5-b3a7-4daa-a2a5-dc3e8a077852/volumes"
Jul 15 15:44:14.362645 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.362565    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"]
Jul 15 15:44:14.363105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.362968    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7db30a-2837-463e-95eb-598acc28d1b7" containerName="authorino"
Jul 15 15:44:14.363105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.362980    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7db30a-2837-463e-95eb-598acc28d1b7" containerName="authorino"
Jul 15 15:44:14.363105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.362995    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" containerName="manager"
Jul 15 15:44:14.363105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.363000    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" containerName="manager"
Jul 15 15:44:14.363105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.363063    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52e99d5-b3a7-4daa-a2a5-dc3e8a077852" containerName="manager"
Jul 15 15:44:14.363105 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.363074    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7db30a-2837-463e-95eb-598acc28d1b7" containerName="authorino"
Jul 15 15:44:14.366099 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.366083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.369047 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.369025    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nsmdv\""
Jul 15 15:44:14.378710 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.378685    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"]
Jul 15 15:44:14.494265 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.494228    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-544rc\" (UniqueName: \"kubernetes.io/projected/fce97a94-e8cf-4726-990a-75adc00740e6-kube-api-access-544rc\") pod \"kuadrant-operator-controller-manager-86878c769f-l6bh9\" (UID: \"fce97a94-e8cf-4726-990a-75adc00740e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.494438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.494348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fce97a94-e8cf-4726-990a-75adc00740e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l6bh9\" (UID: \"fce97a94-e8cf-4726-990a-75adc00740e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.595728 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.595693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fce97a94-e8cf-4726-990a-75adc00740e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l6bh9\" (UID: \"fce97a94-e8cf-4726-990a-75adc00740e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.595898 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.595775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-544rc\" (UniqueName: \"kubernetes.io/projected/fce97a94-e8cf-4726-990a-75adc00740e6-kube-api-access-544rc\") pod \"kuadrant-operator-controller-manager-86878c769f-l6bh9\" (UID: \"fce97a94-e8cf-4726-990a-75adc00740e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.596124 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.596104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fce97a94-e8cf-4726-990a-75adc00740e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l6bh9\" (UID: \"fce97a94-e8cf-4726-990a-75adc00740e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.606456 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.606436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-544rc\" (UniqueName: \"kubernetes.io/projected/fce97a94-e8cf-4726-990a-75adc00740e6-kube-api-access-544rc\") pod \"kuadrant-operator-controller-manager-86878c769f-l6bh9\" (UID: \"fce97a94-e8cf-4726-990a-75adc00740e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.676514 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.676449    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.817927 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.817897    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"]
Jul 15 15:44:14.819848 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:44:14.819823    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce97a94_e8cf_4726_990a_75adc00740e6.slice/crio-d8e65098e072f3e844e1e16134a73bace161cf45769ff3a4f325c8493f56423f WatchSource:0}: Error finding container d8e65098e072f3e844e1e16134a73bace161cf45769ff3a4f325c8493f56423f: Status 404 returned error can't find the container with id d8e65098e072f3e844e1e16134a73bace161cf45769ff3a4f325c8493f56423f
Jul 15 15:44:14.928497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.928412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9" event={"ID":"fce97a94-e8cf-4726-990a-75adc00740e6","Type":"ContainerStarted","Data":"2f75799ba78b6265fca90e085fbffd7286a2fd08ba42ef7e6ddf95f3957ffe10"}
Jul 15 15:44:14.928497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.928475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9" event={"ID":"fce97a94-e8cf-4726-990a-75adc00740e6","Type":"ContainerStarted","Data":"d8e65098e072f3e844e1e16134a73bace161cf45769ff3a4f325c8493f56423f"}
Jul 15 15:44:14.928669 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.928589    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:14.948865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:14.948740    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9" podStartSLOduration=0.948724238 podStartE2EDuration="948.724238ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:44:14.947079724 +0000 UTC m=+1179.748539537" watchObservedRunningTime="2026-07-15 15:44:14.948724238 +0000 UTC m=+1179.750184065"
Jul 15 15:44:25.936196 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:25.936166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6bh9"
Jul 15 15:44:35.729011 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:35.728982    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:44:35.733600 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:35.733580    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:44:36.363417 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:36.363388    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:44:47.562436 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:47.562401    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:44:55.971310 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:44:55.971272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:45:00.142697 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:00.142659    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f"]
Jul 15 15:45:00.146861 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:00.146836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f"
Jul 15 15:45:00.149437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:00.149413    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:45:00.154256 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:00.153975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f"]
Jul 15 15:45:00.157290 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:00.157271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f"
Jul 15 15:45:00.293565 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:00.293544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f"]
Jul 15 15:45:00.294987 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:45:00.294965    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740fa25f_39ce_4af9_bc29_dcc5b74c2048.slice/crio-34b32ce633da36761e632bd9d745c8276afdd35d3a55a6b00a66063fe2af81b2 WatchSource:0}: Error finding container 34b32ce633da36761e632bd9d745c8276afdd35d3a55a6b00a66063fe2af81b2: Status 404 returned error can't find the container with id 34b32ce633da36761e632bd9d745c8276afdd35d3a55a6b00a66063fe2af81b2
Jul 15 15:45:00.297376 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:00.297354    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 15:45:01.184874 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:01.184840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f" event={"ID":"740fa25f-39ce-4af9-bc29-dcc5b74c2048","Type":"ContainerStarted","Data":"34b32ce633da36761e632bd9d745c8276afdd35d3a55a6b00a66063fe2af81b2"}
Jul 15 15:45:03.199374 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:03.199341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f" event={"ID":"740fa25f-39ce-4af9-bc29-dcc5b74c2048","Type":"ContainerStarted","Data":"f0aa0f15a937399d25a18d89a356b7603bace1d9b2da0beca8dca7deac8fcbf0"}
Jul 15 15:45:03.216488 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:03.216440    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f" podStartSLOduration=0.70850152 podStartE2EDuration="3.216426384s" totalImagesPullingTime="2.507924864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:45:03.214492699 +0000 UTC m=+1228.015952515" watchObservedRunningTime="2026-07-15 15:45:03.216426384 +0000 UTC m=+1228.017886201"
Jul 15 15:45:04.212090 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:04.212031    2580 generic.go:356] "Generic (PLEG): container finished" podID="740fa25f-39ce-4af9-bc29-dcc5b74c2048" containerID="f0aa0f15a937399d25a18d89a356b7603bace1d9b2da0beca8dca7deac8fcbf0" exitCode=0
Jul 15 15:45:04.212551 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:04.212153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f" event={"ID":"740fa25f-39ce-4af9-bc29-dcc5b74c2048","Type":"ContainerDied","Data":"f0aa0f15a937399d25a18d89a356b7603bace1d9b2da0beca8dca7deac8fcbf0"}
Jul 15 15:45:05.350735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:05.350709    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f"
Jul 15 15:45:06.223653 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:06.223619    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f"
Jul 15 15:45:06.223653 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:06.223640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735505-vcp6f" event={"ID":"740fa25f-39ce-4af9-bc29-dcc5b74c2048","Type":"ContainerDied","Data":"34b32ce633da36761e632bd9d745c8276afdd35d3a55a6b00a66063fe2af81b2"}
Jul 15 15:45:06.223876 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:06.223688    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34b32ce633da36761e632bd9d745c8276afdd35d3a55a6b00a66063fe2af81b2"
Jul 15 15:45:07.069393 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:07.069357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:45:16.166783 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:16.166737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:45:27.166813 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:27.166777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:45:37.270691 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:37.270658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:45:48.370959 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:48.370926    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:45:58.367852 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:45:58.367812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:46:08.069098 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:46:08.069063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:46:18.769395 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:46:18.769358    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:46:43.166856 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:46:43.166820    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:46:50.771795 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:46:50.771760    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:47:00.586051 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:47:00.585973    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:47:11.066764 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:47:11.066713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:47:20.187592 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:47:20.187557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:47:31.282658 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:47:31.282623    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:47:42.027781 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:47:42.027733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:47:51.693028 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:47:51.692988    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:48:01.986620 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:48:01.986587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:48:10.676253 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:48:10.676215    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:48:20.278771 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:48:20.278715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:48:33.178349 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:48:33.178275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:48:42.470547 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:48:42.470512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:48:51.586191 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:48:51.586159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:49:01.789784 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:01.789738    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:49:14.472384 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:14.472348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:49:23.971677 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:23.971642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:49:35.774393 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:35.774358    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:49:35.778869 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:35.778848    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:49:36.069811 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:36.069781    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:49:47.662315 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:47.662280    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:49:56.462575 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:49:56.462536    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:50:10.669237 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:10.669198    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:50:26.754915 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:26.754881    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:50:36.386285 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.386241    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"]
Jul 15 15:50:36.386685 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.386672    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740fa25f-39ce-4af9-bc29-dcc5b74c2048" containerName="cleanup"
Jul 15 15:50:36.386685 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.386686    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="740fa25f-39ce-4af9-bc29-dcc5b74c2048" containerName="cleanup"
Jul 15 15:50:36.386786 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.386781    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="740fa25f-39ce-4af9-bc29-dcc5b74c2048" containerName="cleanup"
Jul 15 15:50:36.389741 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.389723    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"
Jul 15 15:50:36.392033 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.392010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:50:36.392033 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.392020    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 15:50:36.396984 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.396959    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"]
Jul 15 15:50:36.456016 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.455984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81029a23-86cf-4d2a-8d30-5a8afa13f2c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38132c4d-5h659\" (UID: \"81029a23-86cf-4d2a-8d30-5a8afa13f2c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"
Jul 15 15:50:36.557155 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.557122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81029a23-86cf-4d2a-8d30-5a8afa13f2c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38132c4d-5h659\" (UID: \"81029a23-86cf-4d2a-8d30-5a8afa13f2c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"
Jul 15 15:50:36.557720 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.557699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81029a23-86cf-4d2a-8d30-5a8afa13f2c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38132c4d-5h659\" (UID: \"81029a23-86cf-4d2a-8d30-5a8afa13f2c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"
Jul 15 15:50:36.701066 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.700980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"
Jul 15 15:50:36.835109 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:36.835082    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"]
Jul 15 15:50:36.837161 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:50:36.837136    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81029a23_86cf_4d2a_8d30_5a8afa13f2c9.slice/crio-bb3e9770c13f11dec3a4c950d76771208a3f5d9135125380a2049e192aba7186 WatchSource:0}: Error finding container bb3e9770c13f11dec3a4c950d76771208a3f5d9135125380a2049e192aba7186: Status 404 returned error can't find the container with id bb3e9770c13f11dec3a4c950d76771208a3f5d9135125380a2049e192aba7186
Jul 15 15:50:37.086081 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:37.086051    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" event={"ID":"81029a23-86cf-4d2a-8d30-5a8afa13f2c9","Type":"ContainerStarted","Data":"bb3e9770c13f11dec3a4c950d76771208a3f5d9135125380a2049e192aba7186"}
Jul 15 15:50:38.092820 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:38.092784    2580 generic.go:356] "Generic (PLEG): container finished" podID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerID="44980462b5f30bda18f575c15323560fc2d64a50c2b3b24b1e074726e77beefe" exitCode=7
Jul 15 15:50:38.093228 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:38.092925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" event={"ID":"81029a23-86cf-4d2a-8d30-5a8afa13f2c9","Type":"ContainerDied","Data":"44980462b5f30bda18f575c15323560fc2d64a50c2b3b24b1e074726e77beefe"}
Jul 15 15:50:38.093228 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:38.093098    2580 scope.go:117] "RemoveContainer" containerID="44980462b5f30bda18f575c15323560fc2d64a50c2b3b24b1e074726e77beefe"
Jul 15 15:50:39.101095 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:39.101057    2580 generic.go:356] "Generic (PLEG): container finished" podID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerID="6a11c96797a5a5b511b1f336072d1e8c1bdc7f379276df45e272d2fa1ed347e7" exitCode=7
Jul 15 15:50:39.101512 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:39.101147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" event={"ID":"81029a23-86cf-4d2a-8d30-5a8afa13f2c9","Type":"ContainerDied","Data":"6a11c96797a5a5b511b1f336072d1e8c1bdc7f379276df45e272d2fa1ed347e7"}
Jul 15 15:50:39.101512 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:39.101206    2580 scope.go:117] "RemoveContainer" containerID="44980462b5f30bda18f575c15323560fc2d64a50c2b3b24b1e074726e77beefe"
Jul 15 15:50:39.101512 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:39.101404    2580 scope.go:117] "RemoveContainer" containerID="6a11c96797a5a5b511b1f336072d1e8c1bdc7f379276df45e272d2fa1ed347e7"
Jul 15 15:50:39.101647 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:50:39.101628    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-38132c4d-5h659_odh-ai-gateway-infra(81029a23-86cf-4d2a-8d30-5a8afa13f2c9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9"
Jul 15 15:50:39.671331 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:39.671293    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:50:40.108152 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:40.108124    2580 scope.go:117] "RemoveContainer" containerID="6a11c96797a5a5b511b1f336072d1e8c1bdc7f379276df45e272d2fa1ed347e7"
Jul 15 15:50:40.108611 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:50:40.108389    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-38132c4d-5h659_odh-ai-gateway-infra(81029a23-86cf-4d2a-8d30-5a8afa13f2c9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9"
Jul 15 15:50:51.761718 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:51.761687    2580 scope.go:117] "RemoveContainer" containerID="6a11c96797a5a5b511b1f336072d1e8c1bdc7f379276df45e272d2fa1ed347e7"
Jul 15 15:50:52.174097 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:52.174061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" event={"ID":"81029a23-86cf-4d2a-8d30-5a8afa13f2c9","Type":"ContainerStarted","Data":"5b5f730dcc491effe54a1468199fb1e72c47c4f760527c477ad4335f547d23cd"}
Jul 15 15:50:52.191208 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:52.191162    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" podStartSLOduration=16.191149267 podStartE2EDuration="16.191149267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:50:52.189798457 +0000 UTC m=+1576.991258272" watchObservedRunningTime="2026-07-15 15:50:52.191149267 +0000 UTC m=+1576.992609091"
Jul 15 15:50:53.181557 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:53.181519    2580 generic.go:356] "Generic (PLEG): container finished" podID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerID="5b5f730dcc491effe54a1468199fb1e72c47c4f760527c477ad4335f547d23cd" exitCode=0
Jul 15 15:50:53.181995 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:53.181588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" event={"ID":"81029a23-86cf-4d2a-8d30-5a8afa13f2c9","Type":"ContainerDied","Data":"5b5f730dcc491effe54a1468199fb1e72c47c4f760527c477ad4335f547d23cd"}
Jul 15 15:50:53.181995 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:53.181648    2580 scope.go:117] "RemoveContainer" containerID="6a11c96797a5a5b511b1f336072d1e8c1bdc7f379276df45e272d2fa1ed347e7"
Jul 15 15:50:53.206315 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:53.206293    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"]
Jul 15 15:50:54.325899 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:54.325876    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"
Jul 15 15:50:54.507452 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:54.507374    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81029a23-86cf-4d2a-8d30-5a8afa13f2c9-maas-api-service-ca\") pod \"81029a23-86cf-4d2a-8d30-5a8afa13f2c9\" (UID: \"81029a23-86cf-4d2a-8d30-5a8afa13f2c9\") "
Jul 15 15:50:54.507644 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:54.507623    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81029a23-86cf-4d2a-8d30-5a8afa13f2c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81029a23-86cf-4d2a-8d30-5a8afa13f2c9" (UID: "81029a23-86cf-4d2a-8d30-5a8afa13f2c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:50:54.608808 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:54.608780    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81029a23-86cf-4d2a-8d30-5a8afa13f2c9-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:50:55.197668 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:55.197624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659" event={"ID":"81029a23-86cf-4d2a-8d30-5a8afa13f2c9","Type":"ContainerDied","Data":"bb3e9770c13f11dec3a4c950d76771208a3f5d9135125380a2049e192aba7186"}
Jul 15 15:50:55.197853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:55.197680    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb3e9770c13f11dec3a4c950d76771208a3f5d9135125380a2049e192aba7186"
Jul 15 15:50:55.197853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:55.197640    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"
Jul 15 15:50:55.228462 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:55.228430    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"]
Jul 15 15:50:55.233783 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:55.233737    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-5h659"]
Jul 15 15:50:55.775221 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:50:55.775188    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" path="/var/lib/kubelet/pods/81029a23-86cf-4d2a-8d30-5a8afa13f2c9/volumes"
Jul 15 15:51:26.430175 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.430140    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"]
Jul 15 15:51:26.430639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.430503    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:26.430639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.430514    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:26.430639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.430537    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:26.430639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.430543    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:26.430639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.430609    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:26.430639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.430618    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:26.433789 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.433771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"
Jul 15 15:51:26.436471 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.436447    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:51:26.436604 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.436492    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 15:51:26.443012 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.442985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"]
Jul 15 15:51:26.496300 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.496267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba005b4-cf56-43c4-a60a-678d8518deed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq\" (UID: \"3ba005b4-cf56-43c4-a60a-678d8518deed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"
Jul 15 15:51:26.597455 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.597418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba005b4-cf56-43c4-a60a-678d8518deed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq\" (UID: \"3ba005b4-cf56-43c4-a60a-678d8518deed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"
Jul 15 15:51:26.598034 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.598014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba005b4-cf56-43c4-a60a-678d8518deed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq\" (UID: \"3ba005b4-cf56-43c4-a60a-678d8518deed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"
Jul 15 15:51:26.744672 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.744589    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"
Jul 15 15:51:26.879266 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:26.879240    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"]
Jul 15 15:51:26.881388 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:51:26.881363    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba005b4_cf56_43c4_a60a_678d8518deed.slice/crio-b66740b9dd1046b82462d0a947010a64a11ae9269b4cfc7078e794cd2cd901c4 WatchSource:0}: Error finding container b66740b9dd1046b82462d0a947010a64a11ae9269b4cfc7078e794cd2cd901c4: Status 404 returned error can't find the container with id b66740b9dd1046b82462d0a947010a64a11ae9269b4cfc7078e794cd2cd901c4
Jul 15 15:51:27.373886 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:27.373779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq" event={"ID":"3ba005b4-cf56-43c4-a60a-678d8518deed","Type":"ContainerStarted","Data":"bcda35e544bf10574cc9b2bc55c644cbcbbbe94bbb9fc5d86c5b523a45c372fa"}
Jul 15 15:51:27.373886 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:27.373866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq" event={"ID":"3ba005b4-cf56-43c4-a60a-678d8518deed","Type":"ContainerStarted","Data":"b66740b9dd1046b82462d0a947010a64a11ae9269b4cfc7078e794cd2cd901c4"}
Jul 15 15:51:27.391116 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:27.391068    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq" podStartSLOduration=1.3910543 podStartE2EDuration="1.3910543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:51:27.388208866 +0000 UTC m=+1612.189668683" watchObservedRunningTime="2026-07-15 15:51:27.3910543 +0000 UTC m=+1612.192514126"
Jul 15 15:51:28.381008 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:28.380976    2580 generic.go:356] "Generic (PLEG): container finished" podID="3ba005b4-cf56-43c4-a60a-678d8518deed" containerID="bcda35e544bf10574cc9b2bc55c644cbcbbbe94bbb9fc5d86c5b523a45c372fa" exitCode=0
Jul 15 15:51:28.381364 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:28.381063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq" event={"ID":"3ba005b4-cf56-43c4-a60a-678d8518deed","Type":"ContainerDied","Data":"bcda35e544bf10574cc9b2bc55c644cbcbbbe94bbb9fc5d86c5b523a45c372fa"}
Jul 15 15:51:29.533249 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:29.533222    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"
Jul 15 15:51:29.619880 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:29.619851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba005b4-cf56-43c4-a60a-678d8518deed-maas-api-service-ca\") pod \"3ba005b4-cf56-43c4-a60a-678d8518deed\" (UID: \"3ba005b4-cf56-43c4-a60a-678d8518deed\") "
Jul 15 15:51:29.620201 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:29.620177    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ba005b4-cf56-43c4-a60a-678d8518deed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ba005b4-cf56-43c4-a60a-678d8518deed" (UID: "3ba005b4-cf56-43c4-a60a-678d8518deed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:51:29.720997 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:29.720924    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba005b4-cf56-43c4-a60a-678d8518deed-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:51:30.395921 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:30.395892    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq"
Jul 15 15:51:30.396123 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:30.395941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq" event={"ID":"3ba005b4-cf56-43c4-a60a-678d8518deed","Type":"ContainerDied","Data":"b66740b9dd1046b82462d0a947010a64a11ae9269b4cfc7078e794cd2cd901c4"}
Jul 15 15:51:30.396123 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:30.395987    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b66740b9dd1046b82462d0a947010a64a11ae9269b4cfc7078e794cd2cd901c4"
Jul 15 15:51:39.664712 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:39.664675    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:51:53.642815 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.642744    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"]
Jul 15 15:51:53.643203 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.643188    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:53.643252 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.643207    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:53.643252 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.643223    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba005b4-cf56-43c4-a60a-678d8518deed" containerName="revoke-keys"
Jul 15 15:51:53.643252 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.643229    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba005b4-cf56-43c4-a60a-678d8518deed" containerName="revoke-keys"
Jul 15 15:51:53.643347 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.643308    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba005b4-cf56-43c4-a60a-678d8518deed" containerName="revoke-keys"
Jul 15 15:51:53.643347 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.643318    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81029a23-86cf-4d2a-8d30-5a8afa13f2c9" containerName="revoke-keys"
Jul 15 15:51:53.648777 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.647176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.650481 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.650446    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c7b34001-serving-cert\""
Jul 15 15:51:53.650822 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.650800    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 15:51:53.651086 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.651065    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 15:51:53.651504 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.651481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 15:51:53.659674 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.659649    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"]
Jul 15 15:51:53.742093 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.742058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58319a18-f912-4344-8518-c11b5345030e-maas-api-tls\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.742272 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.742129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58319a18-f912-4344-8518-c11b5345030e-openshift-service-ca\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.742272 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.742146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n68cd\" (UniqueName: \"kubernetes.io/projected/58319a18-f912-4344-8518-c11b5345030e-kube-api-access-n68cd\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.843039 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.843009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58319a18-f912-4344-8518-c11b5345030e-openshift-service-ca\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.843039 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.843043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n68cd\" (UniqueName: \"kubernetes.io/projected/58319a18-f912-4344-8518-c11b5345030e-kube-api-access-n68cd\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.843329 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.843150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58319a18-f912-4344-8518-c11b5345030e-maas-api-tls\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.843936 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.843913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58319a18-f912-4344-8518-c11b5345030e-openshift-service-ca\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.845631 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.845606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58319a18-f912-4344-8518-c11b5345030e-maas-api-tls\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.852882 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.852857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n68cd\" (UniqueName: \"kubernetes.io/projected/58319a18-f912-4344-8518-c11b5345030e-kube-api-access-n68cd\") pod \"maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:53.966284 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:53.966209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:54.117097 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:54.117070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"]
Jul 15 15:51:54.119179 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:51:54.119153    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58319a18_f912_4344_8518_c11b5345030e.slice/crio-0411746c0a4938264253bd400f7dc14ce5a86e6e7b0962fc2982e532de07cacc WatchSource:0}: Error finding container 0411746c0a4938264253bd400f7dc14ce5a86e6e7b0962fc2982e532de07cacc: Status 404 returned error can't find the container with id 0411746c0a4938264253bd400f7dc14ce5a86e6e7b0962fc2982e532de07cacc
Jul 15 15:51:54.120545 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:54.120528    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 15:51:54.532145 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:54.532106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl" event={"ID":"58319a18-f912-4344-8518-c11b5345030e","Type":"ContainerStarted","Data":"0411746c0a4938264253bd400f7dc14ce5a86e6e7b0962fc2982e532de07cacc"}
Jul 15 15:51:56.547375 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:56.547336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl" event={"ID":"58319a18-f912-4344-8518-c11b5345030e","Type":"ContainerStarted","Data":"0c8ae0399ed2b906bcd485ef9f218f37f82de8845dd848ea690c6a1accd568f9"}
Jul 15 15:51:56.547747 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:56.547397    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:51:56.566722 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:56.566648    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl" podStartSLOduration=1.365847226 podStartE2EDuration="3.566630788s" totalImagesPullingTime="2.200783562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:51:56.56641195 +0000 UTC m=+1641.367871790" watchObservedRunningTime="2026-07-15 15:51:56.566630788 +0000 UTC m=+1641.368090603"
Jul 15 15:51:58.040889 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.040851    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"]
Jul 15 15:51:58.044308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.044286    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"
Jul 15 15:51:58.047701 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.047682    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:51:58.053721 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.053698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"]
Jul 15 15:51:58.085356 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.085328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78bc2f1b-e331-4855-aa5d-a9868ed079b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf\" (UID: \"78bc2f1b-e331-4855-aa5d-a9868ed079b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"
Jul 15 15:51:58.185991 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.185961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78bc2f1b-e331-4855-aa5d-a9868ed079b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf\" (UID: \"78bc2f1b-e331-4855-aa5d-a9868ed079b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"
Jul 15 15:51:58.186578 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.186550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78bc2f1b-e331-4855-aa5d-a9868ed079b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf\" (UID: \"78bc2f1b-e331-4855-aa5d-a9868ed079b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"
Jul 15 15:51:58.355598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.355565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"
Jul 15 15:51:58.703199 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:58.703173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"]
Jul 15 15:51:58.704889 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:51:58.704858    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bc2f1b_e331_4855_aa5d_a9868ed079b2.slice/crio-dc00b14ac63fd7f0ae33fc14992c8de41082d455ddd06b818bf87ca4b39f4df3 WatchSource:0}: Error finding container dc00b14ac63fd7f0ae33fc14992c8de41082d455ddd06b818bf87ca4b39f4df3: Status 404 returned error can't find the container with id dc00b14ac63fd7f0ae33fc14992c8de41082d455ddd06b818bf87ca4b39f4df3
Jul 15 15:51:59.567984 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:59.567947    2580 generic.go:356] "Generic (PLEG): container finished" podID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerID="ddf34570f10cd22afe5e37f9c589eeb6f9b479621694e86ddcf8cad5a8217db1" exitCode=7
Jul 15 15:51:59.568446 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:59.568015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf" event={"ID":"78bc2f1b-e331-4855-aa5d-a9868ed079b2","Type":"ContainerDied","Data":"ddf34570f10cd22afe5e37f9c589eeb6f9b479621694e86ddcf8cad5a8217db1"}
Jul 15 15:51:59.568446 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:59.568084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf" event={"ID":"78bc2f1b-e331-4855-aa5d-a9868ed079b2","Type":"ContainerStarted","Data":"dc00b14ac63fd7f0ae33fc14992c8de41082d455ddd06b818bf87ca4b39f4df3"}
Jul 15 15:51:59.568446 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:51:59.568337    2580 scope.go:117] "RemoveContainer" containerID="ddf34570f10cd22afe5e37f9c589eeb6f9b479621694e86ddcf8cad5a8217db1"
Jul 15 15:52:00.575921 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:00.575889    2580 generic.go:356] "Generic (PLEG): container finished" podID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerID="438393be20809973e00fce24a3f5d186c3b2314028d5c96f6034b7555a3d89f3" exitCode=7
Jul 15 15:52:00.576362 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:00.575960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf" event={"ID":"78bc2f1b-e331-4855-aa5d-a9868ed079b2","Type":"ContainerDied","Data":"438393be20809973e00fce24a3f5d186c3b2314028d5c96f6034b7555a3d89f3"}
Jul 15 15:52:00.576362 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:00.576021    2580 scope.go:117] "RemoveContainer" containerID="ddf34570f10cd22afe5e37f9c589eeb6f9b479621694e86ddcf8cad5a8217db1"
Jul 15 15:52:00.576362 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:00.576266    2580 scope.go:117] "RemoveContainer" containerID="438393be20809973e00fce24a3f5d186c3b2314028d5c96f6034b7555a3d89f3"
Jul 15 15:52:00.576547 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:52:00.576529    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf_odh-ai-gateway-infra(78bc2f1b-e331-4855-aa5d-a9868ed079b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2"
Jul 15 15:52:01.583226 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:01.583203    2580 scope.go:117] "RemoveContainer" containerID="438393be20809973e00fce24a3f5d186c3b2314028d5c96f6034b7555a3d89f3"
Jul 15 15:52:01.583651 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:52:01.583459    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf_odh-ai-gateway-infra(78bc2f1b-e331-4855-aa5d-a9868ed079b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2"
Jul 15 15:52:02.061640 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:02.061563    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:52:02.559290 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:02.559263    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:52:12.761670 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:12.761640    2580 scope.go:117] "RemoveContainer" containerID="438393be20809973e00fce24a3f5d186c3b2314028d5c96f6034b7555a3d89f3"
Jul 15 15:52:13.651565 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:13.651532    2580 generic.go:356] "Generic (PLEG): container finished" podID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerID="e3a169e2a69ce59ced3bc22db9fe452cbf7441e85c144ddcbb31231b044bb2d1" exitCode=0
Jul 15 15:52:13.651800 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:13.651605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf" event={"ID":"78bc2f1b-e331-4855-aa5d-a9868ed079b2","Type":"ContainerDied","Data":"e3a169e2a69ce59ced3bc22db9fe452cbf7441e85c144ddcbb31231b044bb2d1"}
Jul 15 15:52:13.651800 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:13.651664    2580 scope.go:117] "RemoveContainer" containerID="438393be20809973e00fce24a3f5d186c3b2314028d5c96f6034b7555a3d89f3"
Jul 15 15:52:14.680516 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:14.680485    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"]
Jul 15 15:52:14.793440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:14.793418    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"
Jul 15 15:52:14.947667 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:14.947588    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78bc2f1b-e331-4855-aa5d-a9868ed079b2-maas-api-service-ca\") pod \"78bc2f1b-e331-4855-aa5d-a9868ed079b2\" (UID: \"78bc2f1b-e331-4855-aa5d-a9868ed079b2\") "
Jul 15 15:52:14.947967 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:14.947948    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78bc2f1b-e331-4855-aa5d-a9868ed079b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78bc2f1b-e331-4855-aa5d-a9868ed079b2" (UID: "78bc2f1b-e331-4855-aa5d-a9868ed079b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:52:15.048803 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:15.048772    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78bc2f1b-e331-4855-aa5d-a9868ed079b2-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:52:15.666217 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:15.666189    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"
Jul 15 15:52:15.666379 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:15.666245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf" event={"ID":"78bc2f1b-e331-4855-aa5d-a9868ed079b2","Type":"ContainerDied","Data":"dc00b14ac63fd7f0ae33fc14992c8de41082d455ddd06b818bf87ca4b39f4df3"}
Jul 15 15:52:15.666379 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:15.666294    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc00b14ac63fd7f0ae33fc14992c8de41082d455ddd06b818bf87ca4b39f4df3"
Jul 15 15:52:15.697683 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:15.697648    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"]
Jul 15 15:52:15.701560 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:15.701534    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-7tnnf"]
Jul 15 15:52:15.768201 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:15.768172    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" path="/var/lib/kubelet/pods/78bc2f1b-e331-4855-aa5d-a9868ed079b2/volumes"
Jul 15 15:52:48.078849 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.078817    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"]
Jul 15 15:52:48.079425 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.079402    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:52:48.079505 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.079428    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:52:48.079505 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.079444    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:52:48.079505 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.079451    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:52:48.079655 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.079584    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:52:48.079655 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.079597    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:52:48.079655 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.079608    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:52:48.082828 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.082808    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"
Jul 15 15:52:48.085642 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.085625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:52:48.089900 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.089876    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"]
Jul 15 15:52:48.150632 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.150601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeffbf26-85a6-495d-a86f-ea91ff3bbf6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft\" (UID: \"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"
Jul 15 15:52:48.251592 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.251565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeffbf26-85a6-495d-a86f-ea91ff3bbf6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft\" (UID: \"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"
Jul 15 15:52:48.252182 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.252164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeffbf26-85a6-495d-a86f-ea91ff3bbf6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft\" (UID: \"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"
Jul 15 15:52:48.394922 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.394855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"
Jul 15 15:52:48.526136 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.526109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"]
Jul 15 15:52:48.529506 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:52:48.529474    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeffbf26_85a6_495d_a86f_ea91ff3bbf6c.slice/crio-00250596cec26a165f3ca55dc03cd959b308d0096bc3ec331d2930fb65547113 WatchSource:0}: Error finding container 00250596cec26a165f3ca55dc03cd959b308d0096bc3ec331d2930fb65547113: Status 404 returned error can't find the container with id 00250596cec26a165f3ca55dc03cd959b308d0096bc3ec331d2930fb65547113
Jul 15 15:52:48.850297 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.850266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft" event={"ID":"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c","Type":"ContainerStarted","Data":"7292e09bb774e167c61abd0e21ccba2ae66d1b59ad295149c9e4595349d48681"}
Jul 15 15:52:48.850401 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.850318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft" event={"ID":"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c","Type":"ContainerStarted","Data":"00250596cec26a165f3ca55dc03cd959b308d0096bc3ec331d2930fb65547113"}
Jul 15 15:52:48.867643 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:48.867605    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft" podStartSLOduration=0.867590627 podStartE2EDuration="867.590627ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:52:48.86645585 +0000 UTC m=+1693.667915659" watchObservedRunningTime="2026-07-15 15:52:48.867590627 +0000 UTC m=+1693.669050442"
Jul 15 15:52:49.858106 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:49.858069    2580 generic.go:356] "Generic (PLEG): container finished" podID="aeffbf26-85a6-495d-a86f-ea91ff3bbf6c" containerID="7292e09bb774e167c61abd0e21ccba2ae66d1b59ad295149c9e4595349d48681" exitCode=0
Jul 15 15:52:49.858523 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:49.858157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft" event={"ID":"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c","Type":"ContainerDied","Data":"7292e09bb774e167c61abd0e21ccba2ae66d1b59ad295149c9e4595349d48681"}
Jul 15 15:52:50.989139 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:50.989108    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"
Jul 15 15:52:51.077420 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:51.077393    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeffbf26-85a6-495d-a86f-ea91ff3bbf6c-maas-api-service-ca\") pod \"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c\" (UID: \"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c\") "
Jul 15 15:52:51.077793 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:51.077741    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeffbf26-85a6-495d-a86f-ea91ff3bbf6c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aeffbf26-85a6-495d-a86f-ea91ff3bbf6c" (UID: "aeffbf26-85a6-495d-a86f-ea91ff3bbf6c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:52:51.178090 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:51.178024    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeffbf26-85a6-495d-a86f-ea91ff3bbf6c-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:52:51.871053 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:51.871026    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft"
Jul 15 15:52:51.871233 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:51.871025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft" event={"ID":"aeffbf26-85a6-495d-a86f-ea91ff3bbf6c","Type":"ContainerDied","Data":"00250596cec26a165f3ca55dc03cd959b308d0096bc3ec331d2930fb65547113"}
Jul 15 15:52:51.871233 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:51.871140    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00250596cec26a165f3ca55dc03cd959b308d0096bc3ec331d2930fb65547113"
Jul 15 15:52:58.674675 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:58.674601    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"]
Jul 15 15:52:58.675060 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:58.674839    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl" podUID="58319a18-f912-4344-8518-c11b5345030e" containerName="maas-api" containerID="cri-o://0c8ae0399ed2b906bcd485ef9f218f37f82de8845dd848ea690c6a1accd568f9" gracePeriod=30
Jul 15 15:52:58.915472 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:58.915444    2580 generic.go:356] "Generic (PLEG): container finished" podID="58319a18-f912-4344-8518-c11b5345030e" containerID="0c8ae0399ed2b906bcd485ef9f218f37f82de8845dd848ea690c6a1accd568f9" exitCode=0
Jul 15 15:52:58.915604 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:58.915496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl" event={"ID":"58319a18-f912-4344-8518-c11b5345030e","Type":"ContainerDied","Data":"0c8ae0399ed2b906bcd485ef9f218f37f82de8845dd848ea690c6a1accd568f9"}
Jul 15 15:52:58.932023 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:58.931968    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:52:59.049622 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.049596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n68cd\" (UniqueName: \"kubernetes.io/projected/58319a18-f912-4344-8518-c11b5345030e-kube-api-access-n68cd\") pod \"58319a18-f912-4344-8518-c11b5345030e\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") "
Jul 15 15:52:59.049795 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.049647    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58319a18-f912-4344-8518-c11b5345030e-maas-api-tls\") pod \"58319a18-f912-4344-8518-c11b5345030e\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") "
Jul 15 15:52:59.049795 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.049705    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58319a18-f912-4344-8518-c11b5345030e-openshift-service-ca\") pod \"58319a18-f912-4344-8518-c11b5345030e\" (UID: \"58319a18-f912-4344-8518-c11b5345030e\") "
Jul 15 15:52:59.050125 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.050101    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58319a18-f912-4344-8518-c11b5345030e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58319a18-f912-4344-8518-c11b5345030e" (UID: "58319a18-f912-4344-8518-c11b5345030e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:52:59.051790 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.051741    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58319a18-f912-4344-8518-c11b5345030e-kube-api-access-n68cd" (OuterVolumeSpecName: "kube-api-access-n68cd") pod "58319a18-f912-4344-8518-c11b5345030e" (UID: "58319a18-f912-4344-8518-c11b5345030e"). InnerVolumeSpecName "kube-api-access-n68cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:52:59.052192 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.052164    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58319a18-f912-4344-8518-c11b5345030e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58319a18-f912-4344-8518-c11b5345030e" (UID: "58319a18-f912-4344-8518-c11b5345030e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:52:59.151399 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.151373    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58319a18-f912-4344-8518-c11b5345030e-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:52:59.151399 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.151399    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n68cd\" (UniqueName: \"kubernetes.io/projected/58319a18-f912-4344-8518-c11b5345030e-kube-api-access-n68cd\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:52:59.151571 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.151409    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58319a18-f912-4344-8518-c11b5345030e-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:52:59.924498 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.924462    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"
Jul 15 15:52:59.924947 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.924482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl" event={"ID":"58319a18-f912-4344-8518-c11b5345030e","Type":"ContainerDied","Data":"0411746c0a4938264253bd400f7dc14ce5a86e6e7b0962fc2982e532de07cacc"}
Jul 15 15:52:59.924947 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.924596    2580 scope.go:117] "RemoveContainer" containerID="0c8ae0399ed2b906bcd485ef9f218f37f82de8845dd848ea690c6a1accd568f9"
Jul 15 15:52:59.944427 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.944401    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"]
Jul 15 15:52:59.948652 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:52:59.948626    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c7b34001-7ddbc5784c-9f8xl"]
Jul 15 15:53:01.166937 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:01.166897    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:53:01.767352 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:01.767315    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58319a18-f912-4344-8518-c11b5345030e" path="/var/lib/kubelet/pods/58319a18-f912-4344-8518-c11b5345030e/volumes"
Jul 15 15:53:05.820710 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.820680    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"]
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821068    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821079    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78bc2f1b-e331-4855-aa5d-a9868ed079b2" containerName="revoke-keys"
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821099    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeffbf26-85a6-495d-a86f-ea91ff3bbf6c" containerName="revoke-keys"
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821105    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeffbf26-85a6-495d-a86f-ea91ff3bbf6c" containerName="revoke-keys"
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821119    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58319a18-f912-4344-8518-c11b5345030e" containerName="maas-api"
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821124    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="58319a18-f912-4344-8518-c11b5345030e" containerName="maas-api"
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821181    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="58319a18-f912-4344-8518-c11b5345030e" containerName="maas-api"
Jul 15 15:53:05.821187 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.821194    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeffbf26-85a6-495d-a86f-ea91ff3bbf6c" containerName="revoke-keys"
Jul 15 15:53:05.831821 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.831792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:05.837715 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.837689    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 15:53:05.838143 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.838115    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 15:53:05.838362 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.838146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 15:53:05.838362 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.838325    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c020b6bf-serving-cert\""
Jul 15 15:53:05.840348 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:05.840325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"]
Jul 15 15:53:06.011597 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.011554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-openshift-service-ca\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.011774 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.011620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-maas-api-tls\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.011774 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.011653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bckt4\" (UniqueName: \"kubernetes.io/projected/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-kube-api-access-bckt4\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.113026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.112948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-openshift-service-ca\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.113026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.112997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-maas-api-tls\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.113026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.113017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bckt4\" (UniqueName: \"kubernetes.io/projected/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-kube-api-access-bckt4\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.114002 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.113972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-openshift-service-ca\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.115659 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.115639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-maas-api-tls\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.127029 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.127009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bckt4\" (UniqueName: \"kubernetes.io/projected/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-kube-api-access-bckt4\") pod \"maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.146965 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.146938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:06.287388 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.287360    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"]
Jul 15 15:53:06.288534 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:53:06.288499    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2afcc4_dfd6_4d14_ba4e_ca25593e6997.slice/crio-6ec581341a5d7417e8691c7e6cb4b83f52fcf73c7bce356fa5a5ac37a1109c84 WatchSource:0}: Error finding container 6ec581341a5d7417e8691c7e6cb4b83f52fcf73c7bce356fa5a5ac37a1109c84: Status 404 returned error can't find the container with id 6ec581341a5d7417e8691c7e6cb4b83f52fcf73c7bce356fa5a5ac37a1109c84
Jul 15 15:53:06.966158 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:06.966126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8" event={"ID":"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997","Type":"ContainerStarted","Data":"6ec581341a5d7417e8691c7e6cb4b83f52fcf73c7bce356fa5a5ac37a1109c84"}
Jul 15 15:53:08.979246 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:08.979207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8" event={"ID":"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997","Type":"ContainerStarted","Data":"1397a83b851c4e03fcf7f7434166a3608a9fc0f484c74bc16b2a7498defe45bc"}
Jul 15 15:53:08.979682 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:08.979270    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:08.998548 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:08.998501    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8" podStartSLOduration=1.8631354230000001 podStartE2EDuration="3.998485294s" totalImagesPullingTime="2.135349871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:53:08.996684709 +0000 UTC m=+1713.798144533" watchObservedRunningTime="2026-07-15 15:53:08.998485294 +0000 UTC m=+1713.799945110"
Jul 15 15:53:10.645905 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.645868    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"]
Jul 15 15:53:10.651128 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.651099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"
Jul 15 15:53:10.654265 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.654238    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:53:10.655215 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.655184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2adbf64e-1c7f-4783-a40f-b2990e5e02b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98\" (UID: \"2adbf64e-1c7f-4783-a40f-b2990e5e02b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"
Jul 15 15:53:10.656364 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.656341    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"]
Jul 15 15:53:10.756264 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.756231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2adbf64e-1c7f-4783-a40f-b2990e5e02b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98\" (UID: \"2adbf64e-1c7f-4783-a40f-b2990e5e02b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"
Jul 15 15:53:10.756836 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.756819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2adbf64e-1c7f-4783-a40f-b2990e5e02b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98\" (UID: \"2adbf64e-1c7f-4783-a40f-b2990e5e02b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"
Jul 15 15:53:10.963807 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:10.963715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"
Jul 15 15:53:11.100953 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:11.100919    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"]
Jul 15 15:53:11.103357 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:53:11.103325    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2adbf64e_1c7f_4783_a40f_b2990e5e02b2.slice/crio-f27f364a4a82097a3c9e2baa66c456d93f39d5c39a6a36c6eb33f13694a95d88 WatchSource:0}: Error finding container f27f364a4a82097a3c9e2baa66c456d93f39d5c39a6a36c6eb33f13694a95d88: Status 404 returned error can't find the container with id f27f364a4a82097a3c9e2baa66c456d93f39d5c39a6a36c6eb33f13694a95d88
Jul 15 15:53:11.999700 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:11.999610    2580 generic.go:356] "Generic (PLEG): container finished" podID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerID="df5bc09c110c23b2170a684c561f71a33654a1900a7ae08ea045447d7598ac34" exitCode=7
Jul 15 15:53:12.000084 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:11.999693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" event={"ID":"2adbf64e-1c7f-4783-a40f-b2990e5e02b2","Type":"ContainerDied","Data":"df5bc09c110c23b2170a684c561f71a33654a1900a7ae08ea045447d7598ac34"}
Jul 15 15:53:12.000084 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:11.999746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" event={"ID":"2adbf64e-1c7f-4783-a40f-b2990e5e02b2","Type":"ContainerStarted","Data":"f27f364a4a82097a3c9e2baa66c456d93f39d5c39a6a36c6eb33f13694a95d88"}
Jul 15 15:53:12.000084 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:11.999896    2580 scope.go:117] "RemoveContainer" containerID="df5bc09c110c23b2170a684c561f71a33654a1900a7ae08ea045447d7598ac34"
Jul 15 15:53:12.769769 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:12.769722    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:53:13.008181 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:13.008080    2580 generic.go:356] "Generic (PLEG): container finished" podID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerID="36eb310ddd22b045d88e9c65e5f97392dd3a4652a233cbaa0ea494465fdf13b3" exitCode=7
Jul 15 15:53:13.008608 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:13.008173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" event={"ID":"2adbf64e-1c7f-4783-a40f-b2990e5e02b2","Type":"ContainerDied","Data":"36eb310ddd22b045d88e9c65e5f97392dd3a4652a233cbaa0ea494465fdf13b3"}
Jul 15 15:53:13.008608 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:13.008253    2580 scope.go:117] "RemoveContainer" containerID="df5bc09c110c23b2170a684c561f71a33654a1900a7ae08ea045447d7598ac34"
Jul 15 15:53:13.008608 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:13.008435    2580 scope.go:117] "RemoveContainer" containerID="36eb310ddd22b045d88e9c65e5f97392dd3a4652a233cbaa0ea494465fdf13b3"
Jul 15 15:53:13.008778 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:53:13.008672    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98_odh-ai-gateway-infra(2adbf64e-1c7f-4783-a40f-b2990e5e02b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2"
Jul 15 15:53:14.016446 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:14.016416    2580 scope.go:117] "RemoveContainer" containerID="36eb310ddd22b045d88e9c65e5f97392dd3a4652a233cbaa0ea494465fdf13b3"
Jul 15 15:53:14.016832 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:53:14.016604    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98_odh-ai-gateway-infra(2adbf64e-1c7f-4783-a40f-b2990e5e02b2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2"
Jul 15 15:53:14.991113 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:14.991084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:53:28.761488 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:28.761459    2580 scope.go:117] "RemoveContainer" containerID="36eb310ddd22b045d88e9c65e5f97392dd3a4652a233cbaa0ea494465fdf13b3"
Jul 15 15:53:29.104850 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:29.104813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" event={"ID":"2adbf64e-1c7f-4783-a40f-b2990e5e02b2","Type":"ContainerStarted","Data":"09a55aa9f26a746acacf60c463b98ad3c23bcc0d3f901d00256823853ae693d0"}
Jul 15 15:53:29.124410 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:29.124356    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" podStartSLOduration=19.124342283 podStartE2EDuration="19.124342283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:53:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:53:29.121643903 +0000 UTC m=+1733.923103729" watchObservedRunningTime="2026-07-15 15:53:29.124342283 +0000 UTC m=+1733.925802138"
Jul 15 15:53:30.112989 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:30.112950    2580 generic.go:356] "Generic (PLEG): container finished" podID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerID="09a55aa9f26a746acacf60c463b98ad3c23bcc0d3f901d00256823853ae693d0" exitCode=0
Jul 15 15:53:30.113438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:30.113022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" event={"ID":"2adbf64e-1c7f-4783-a40f-b2990e5e02b2","Type":"ContainerDied","Data":"09a55aa9f26a746acacf60c463b98ad3c23bcc0d3f901d00256823853ae693d0"}
Jul 15 15:53:30.113438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:30.113079    2580 scope.go:117] "RemoveContainer" containerID="36eb310ddd22b045d88e9c65e5f97392dd3a4652a233cbaa0ea494465fdf13b3"
Jul 15 15:53:30.144251 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:30.144222    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"]
Jul 15 15:53:31.254642 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:31.254621    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"
Jul 15 15:53:31.337833 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:31.337806    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2adbf64e-1c7f-4783-a40f-b2990e5e02b2-maas-api-service-ca\") pod \"2adbf64e-1c7f-4783-a40f-b2990e5e02b2\" (UID: \"2adbf64e-1c7f-4783-a40f-b2990e5e02b2\") "
Jul 15 15:53:31.338138 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:31.338116    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2adbf64e-1c7f-4783-a40f-b2990e5e02b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2adbf64e-1c7f-4783-a40f-b2990e5e02b2" (UID: "2adbf64e-1c7f-4783-a40f-b2990e5e02b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:53:31.439250 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:31.439175    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2adbf64e-1c7f-4783-a40f-b2990e5e02b2-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:53:32.127895 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:32.127867    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"
Jul 15 15:53:32.128083 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:32.127867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98" event={"ID":"2adbf64e-1c7f-4783-a40f-b2990e5e02b2","Type":"ContainerDied","Data":"f27f364a4a82097a3c9e2baa66c456d93f39d5c39a6a36c6eb33f13694a95d88"}
Jul 15 15:53:32.128083 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:32.127999    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f27f364a4a82097a3c9e2baa66c456d93f39d5c39a6a36c6eb33f13694a95d88"
Jul 15 15:53:32.158445 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:32.158416    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"]
Jul 15 15:53:32.164284 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:32.164259    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-gwl98"]
Jul 15 15:53:33.767977 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:53:33.767946    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" path="/var/lib/kubelet/pods/2adbf64e-1c7f-4783-a40f-b2990e5e02b2/volumes"
Jul 15 15:54:10.696028 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.695998    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"]
Jul 15 15:54:10.696495 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696395    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.696495 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696408    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.696495 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696414    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.696495 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696420    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.696495 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696437    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.696495 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696442    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.696689 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696503    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.696689 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.696512    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:10.699029 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.699012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"
Jul 15 15:54:10.701569 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.701546    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:54:10.707135 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.707113    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"]
Jul 15 15:54:10.790058 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.790026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac752e5-7849-4ecd-8100-2a8713888c70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7\" (UID: \"2ac752e5-7849-4ecd-8100-2a8713888c70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"
Jul 15 15:54:10.891081 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.891049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac752e5-7849-4ecd-8100-2a8713888c70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7\" (UID: \"2ac752e5-7849-4ecd-8100-2a8713888c70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"
Jul 15 15:54:10.891650 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:10.891633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac752e5-7849-4ecd-8100-2a8713888c70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7\" (UID: \"2ac752e5-7849-4ecd-8100-2a8713888c70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"
Jul 15 15:54:11.010912 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:11.010839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"
Jul 15 15:54:11.171028 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:11.171004    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"]
Jul 15 15:54:11.173176 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:54:11.173149    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac752e5_7849_4ecd_8100_2a8713888c70.slice/crio-e18b1fd7962267c38b31304cf6ca32325bc1b0757911fc9c86afa23e015487d7 WatchSource:0}: Error finding container e18b1fd7962267c38b31304cf6ca32325bc1b0757911fc9c86afa23e015487d7: Status 404 returned error can't find the container with id e18b1fd7962267c38b31304cf6ca32325bc1b0757911fc9c86afa23e015487d7
Jul 15 15:54:11.350082 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:11.350048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7" event={"ID":"2ac752e5-7849-4ecd-8100-2a8713888c70","Type":"ContainerStarted","Data":"e18b1fd7962267c38b31304cf6ca32325bc1b0757911fc9c86afa23e015487d7"}
Jul 15 15:54:12.356662 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:12.356630    2580 generic.go:356] "Generic (PLEG): container finished" podID="2ac752e5-7849-4ecd-8100-2a8713888c70" containerID="727600db90cf13b457b315de0e4c81714358f8e3c7e099b2a1867a25c4d680b5" exitCode=0
Jul 15 15:54:12.357046 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:12.356716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7" event={"ID":"2ac752e5-7849-4ecd-8100-2a8713888c70","Type":"ContainerDied","Data":"727600db90cf13b457b315de0e4c81714358f8e3c7e099b2a1867a25c4d680b5"}
Jul 15 15:54:13.504825 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:13.504795    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"
Jul 15 15:54:13.618908 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:13.618835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac752e5-7849-4ecd-8100-2a8713888c70-maas-api-service-ca\") pod \"2ac752e5-7849-4ecd-8100-2a8713888c70\" (UID: \"2ac752e5-7849-4ecd-8100-2a8713888c70\") "
Jul 15 15:54:13.619121 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:13.619098    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac752e5-7849-4ecd-8100-2a8713888c70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ac752e5-7849-4ecd-8100-2a8713888c70" (UID: "2ac752e5-7849-4ecd-8100-2a8713888c70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:54:13.720227 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:13.720191    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac752e5-7849-4ecd-8100-2a8713888c70-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:54:14.369655 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:14.369623    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7"
Jul 15 15:54:14.369849 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:14.369673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7" event={"ID":"2ac752e5-7849-4ecd-8100-2a8713888c70","Type":"ContainerDied","Data":"e18b1fd7962267c38b31304cf6ca32325bc1b0757911fc9c86afa23e015487d7"}
Jul 15 15:54:14.369849 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:14.369725    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e18b1fd7962267c38b31304cf6ca32325bc1b0757911fc9c86afa23e015487d7"
Jul 15 15:54:21.292701 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.292668    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"]
Jul 15 15:54:21.293201 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.292989    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8" podUID="5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" containerName="maas-api" containerID="cri-o://1397a83b851c4e03fcf7f7434166a3608a9fc0f484c74bc16b2a7498defe45bc" gracePeriod=30
Jul 15 15:54:21.413790 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.413742    2580 generic.go:356] "Generic (PLEG): container finished" podID="5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" containerID="1397a83b851c4e03fcf7f7434166a3608a9fc0f484c74bc16b2a7498defe45bc" exitCode=0
Jul 15 15:54:21.413928 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.413822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8" event={"ID":"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997","Type":"ContainerDied","Data":"1397a83b851c4e03fcf7f7434166a3608a9fc0f484c74bc16b2a7498defe45bc"}
Jul 15 15:54:21.542389 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.542365    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:54:21.690600 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.690568    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-openshift-service-ca\") pod \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") "
Jul 15 15:54:21.690600 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.690602    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-maas-api-tls\") pod \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") "
Jul 15 15:54:21.690883 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.690671    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bckt4\" (UniqueName: \"kubernetes.io/projected/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-kube-api-access-bckt4\") pod \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\" (UID: \"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997\") "
Jul 15 15:54:21.690975 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.690951    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" (UID: "5b2afcc4-dfd6-4d14-ba4e-ca25593e6997"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:54:21.691037 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.691022    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:54:21.692773 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.692738    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" (UID: "5b2afcc4-dfd6-4d14-ba4e-ca25593e6997"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:54:21.692773 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.692742    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-kube-api-access-bckt4" (OuterVolumeSpecName: "kube-api-access-bckt4") pod "5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" (UID: "5b2afcc4-dfd6-4d14-ba4e-ca25593e6997"). InnerVolumeSpecName "kube-api-access-bckt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:54:21.791581 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.791554    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bckt4\" (UniqueName: \"kubernetes.io/projected/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-kube-api-access-bckt4\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:54:21.791581 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:21.791578    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:54:22.421171 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:22.421138    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"
Jul 15 15:54:22.421584 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:22.421189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8" event={"ID":"5b2afcc4-dfd6-4d14-ba4e-ca25593e6997","Type":"ContainerDied","Data":"6ec581341a5d7417e8691c7e6cb4b83f52fcf73c7bce356fa5a5ac37a1109c84"}
Jul 15 15:54:22.421584 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:22.421251    2580 scope.go:117] "RemoveContainer" containerID="1397a83b851c4e03fcf7f7434166a3608a9fc0f484c74bc16b2a7498defe45bc"
Jul 15 15:54:22.444710 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:22.444685    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"]
Jul 15 15:54:22.454006 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:22.453982    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c020b6bf-6b5d87dd47-tgns8"]
Jul 15 15:54:23.768208 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:23.768174    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" path="/var/lib/kubelet/pods/5b2afcc4-dfd6-4d14-ba4e-ca25593e6997/volumes"
Jul 15 15:54:23.968419 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:23.968386    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:54:35.813035 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:35.813004    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:54:35.819062 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:35.819042    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:54:38.544792 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.544739    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"]
Jul 15 15:54:38.545366 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.545347    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac752e5-7849-4ecd-8100-2a8713888c70" containerName="revoke-keys"
Jul 15 15:54:38.545437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.545370    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac752e5-7849-4ecd-8100-2a8713888c70" containerName="revoke-keys"
Jul 15 15:54:38.545437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.545388    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" containerName="maas-api"
Jul 15 15:54:38.545437 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.545396    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" containerName="maas-api"
Jul 15 15:54:38.545549 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.545475    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2adbf64e-1c7f-4783-a40f-b2990e5e02b2" containerName="revoke-keys"
Jul 15 15:54:38.545549 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.545486    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac752e5-7849-4ecd-8100-2a8713888c70" containerName="revoke-keys"
Jul 15 15:54:38.545549 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.545495    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2afcc4-dfd6-4d14-ba4e-ca25593e6997" containerName="maas-api"
Jul 15 15:54:38.548462 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.548446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.552736 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.552713    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 15:54:38.553176 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.553159    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4d69cf00-serving-cert\""
Jul 15 15:54:38.553263 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.553178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 15:54:38.553263 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.553210    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 15:54:38.565119 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.565098    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"]
Jul 15 15:54:38.649960 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.649930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-openshift-service-ca\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.650138 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.649981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.650208 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.650134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlpn\" (UniqueName: \"kubernetes.io/projected/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-kube-api-access-jtlpn\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.750964 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.750930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlpn\" (UniqueName: \"kubernetes.io/projected/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-kube-api-access-jtlpn\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.751151 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.750991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-openshift-service-ca\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.751151 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.751033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.751151 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:54:38.751143    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-serving-cert: secret "maas-api-e2e-derive-4d69cf00-serving-cert" not found
Jul 15 15:54:38.751311 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:54:38.751220    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls podName:cf6ca770-eff0-4c1e-892b-bc5daf7a6259 nodeName:}" failed. No retries permitted until 2026-07-15 15:54:39.251199246 +0000 UTC m=+1804.052659038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls") pod "maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6" (UID: "cf6ca770-eff0-4c1e-892b-bc5daf7a6259") : secret "maas-api-e2e-derive-4d69cf00-serving-cert" not found
Jul 15 15:54:38.751777 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.751727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-openshift-service-ca\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:38.761440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:38.761411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlpn\" (UniqueName: \"kubernetes.io/projected/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-kube-api-access-jtlpn\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:39.254051 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:39.254016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:39.256523 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:39.256495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls\") pod \"maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:39.459630 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:39.459599    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:39.604150 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:39.604125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"]
Jul 15 15:54:39.605475 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:54:39.605449    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6ca770_eff0_4c1e_892b_bc5daf7a6259.slice/crio-4f0faf767a0a25182fe242f6fbb39bf8ae337cb978cdccffc8e3b9d5a6fa7e71 WatchSource:0}: Error finding container 4f0faf767a0a25182fe242f6fbb39bf8ae337cb978cdccffc8e3b9d5a6fa7e71: Status 404 returned error can't find the container with id 4f0faf767a0a25182fe242f6fbb39bf8ae337cb978cdccffc8e3b9d5a6fa7e71
Jul 15 15:54:40.536613 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:40.536513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6" event={"ID":"cf6ca770-eff0-4c1e-892b-bc5daf7a6259","Type":"ContainerStarted","Data":"dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069"}
Jul 15 15:54:40.536613 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:40.536585    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:40.536613 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:40.536607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6" event={"ID":"cf6ca770-eff0-4c1e-892b-bc5daf7a6259","Type":"ContainerStarted","Data":"4f0faf767a0a25182fe242f6fbb39bf8ae337cb978cdccffc8e3b9d5a6fa7e71"}
Jul 15 15:54:40.557731 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:40.557685    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6" podStartSLOduration=1.998108752 podStartE2EDuration="2.557670682s" totalImagesPullingTime="559.56193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:54:40.555117002 +0000 UTC m=+1805.356576817" watchObservedRunningTime="2026-07-15 15:54:40.557670682 +0000 UTC m=+1805.359130537"
Jul 15 15:54:42.659958 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.659922    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"]
Jul 15 15:54:42.663703 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.663687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"
Jul 15 15:54:42.666530 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.666515    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:54:42.672337 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.672305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"]
Jul 15 15:54:42.785775 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.785715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d0a192-5070-4e4a-abb7-bfc39a336676-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk\" (UID: \"10d0a192-5070-4e4a-abb7-bfc39a336676\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"
Jul 15 15:54:42.886806 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.886774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d0a192-5070-4e4a-abb7-bfc39a336676-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk\" (UID: \"10d0a192-5070-4e4a-abb7-bfc39a336676\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"
Jul 15 15:54:42.887317 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.887296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d0a192-5070-4e4a-abb7-bfc39a336676-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk\" (UID: \"10d0a192-5070-4e4a-abb7-bfc39a336676\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"
Jul 15 15:54:42.975443 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:42.975370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"
Jul 15 15:54:43.111762 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:43.111724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"]
Jul 15 15:54:43.114231 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:54:43.114201    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d0a192_5070_4e4a_abb7_bfc39a336676.slice/crio-584b08c925936c423de4009e4a41b7523315abc63b4ab1cf3dc5e355752d86e5 WatchSource:0}: Error finding container 584b08c925936c423de4009e4a41b7523315abc63b4ab1cf3dc5e355752d86e5: Status 404 returned error can't find the container with id 584b08c925936c423de4009e4a41b7523315abc63b4ab1cf3dc5e355752d86e5
Jul 15 15:54:43.554639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:43.554535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" event={"ID":"10d0a192-5070-4e4a-abb7-bfc39a336676","Type":"ContainerStarted","Data":"a8f5570ff8b7a659f6ab59f12d5f3eed7aa66891cffc9d4245c106931b9871a8"}
Jul 15 15:54:43.554639 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:43.554602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" event={"ID":"10d0a192-5070-4e4a-abb7-bfc39a336676","Type":"ContainerStarted","Data":"584b08c925936c423de4009e4a41b7523315abc63b4ab1cf3dc5e355752d86e5"}
Jul 15 15:54:43.573297 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:43.573250    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" podStartSLOduration=1.573231517 podStartE2EDuration="1.573231517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:54:43.57154995 +0000 UTC m=+1808.373009765" watchObservedRunningTime="2026-07-15 15:54:43.573231517 +0000 UTC m=+1808.374691330"
Jul 15 15:54:44.561846 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:44.561811    2580 generic.go:356] "Generic (PLEG): container finished" podID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerID="a8f5570ff8b7a659f6ab59f12d5f3eed7aa66891cffc9d4245c106931b9871a8" exitCode=7
Jul 15 15:54:44.562226 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:44.561897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" event={"ID":"10d0a192-5070-4e4a-abb7-bfc39a336676","Type":"ContainerDied","Data":"a8f5570ff8b7a659f6ab59f12d5f3eed7aa66891cffc9d4245c106931b9871a8"}
Jul 15 15:54:44.562226 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:44.562081    2580 scope.go:117] "RemoveContainer" containerID="a8f5570ff8b7a659f6ab59f12d5f3eed7aa66891cffc9d4245c106931b9871a8"
Jul 15 15:54:45.570354 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:45.570324    2580 generic.go:356] "Generic (PLEG): container finished" podID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerID="6c692c76638ef4e328d9c2230752d0ac577bc5032617387cc98144621787b45a" exitCode=7
Jul 15 15:54:45.570741 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:45.570379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" event={"ID":"10d0a192-5070-4e4a-abb7-bfc39a336676","Type":"ContainerDied","Data":"6c692c76638ef4e328d9c2230752d0ac577bc5032617387cc98144621787b45a"}
Jul 15 15:54:45.570741 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:45.570427    2580 scope.go:117] "RemoveContainer" containerID="a8f5570ff8b7a659f6ab59f12d5f3eed7aa66891cffc9d4245c106931b9871a8"
Jul 15 15:54:45.570741 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:45.570612    2580 scope.go:117] "RemoveContainer" containerID="6c692c76638ef4e328d9c2230752d0ac577bc5032617387cc98144621787b45a"
Jul 15 15:54:45.570968 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:54:45.570876    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk_odh-ai-gateway-infra(10d0a192-5070-4e4a-abb7-bfc39a336676)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676"
Jul 15 15:54:45.770834 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:45.770806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:54:46.547589 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:46.547562    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:54:46.577876 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:46.577852    2580 scope.go:117] "RemoveContainer" containerID="6c692c76638ef4e328d9c2230752d0ac577bc5032617387cc98144621787b45a"
Jul 15 15:54:46.578229 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:54:46.578064    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk_odh-ai-gateway-infra(10d0a192-5070-4e4a-abb7-bfc39a336676)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676"
Jul 15 15:54:58.762047 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:58.762013    2580 scope.go:117] "RemoveContainer" containerID="6c692c76638ef4e328d9c2230752d0ac577bc5032617387cc98144621787b45a"
Jul 15 15:54:59.657911 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:59.657878    2580 generic.go:356] "Generic (PLEG): container finished" podID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerID="4791e9f712ba7cbd33ba12be699a3f9bd4d52d9af392f26dd810842c3a19cae3" exitCode=0
Jul 15 15:54:59.658080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:59.657958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" event={"ID":"10d0a192-5070-4e4a-abb7-bfc39a336676","Type":"ContainerDied","Data":"4791e9f712ba7cbd33ba12be699a3f9bd4d52d9af392f26dd810842c3a19cae3"}
Jul 15 15:54:59.658080 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:54:59.658024    2580 scope.go:117] "RemoveContainer" containerID="6c692c76638ef4e328d9c2230752d0ac577bc5032617387cc98144621787b45a"
Jul 15 15:55:00.695223 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:00.695192    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"]
Jul 15 15:55:00.808428 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:00.808398    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"
Jul 15 15:55:00.948190 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:00.948102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d0a192-5070-4e4a-abb7-bfc39a336676-maas-api-service-ca\") pod \"10d0a192-5070-4e4a-abb7-bfc39a336676\" (UID: \"10d0a192-5070-4e4a-abb7-bfc39a336676\") "
Jul 15 15:55:00.948448 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:00.948424    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10d0a192-5070-4e4a-abb7-bfc39a336676-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "10d0a192-5070-4e4a-abb7-bfc39a336676" (UID: "10d0a192-5070-4e4a-abb7-bfc39a336676"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:55:01.048924 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:01.048889    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d0a192-5070-4e4a-abb7-bfc39a336676-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:55:01.673497 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:01.673467    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"
Jul 15 15:55:01.673674 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:01.673466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk" event={"ID":"10d0a192-5070-4e4a-abb7-bfc39a336676","Type":"ContainerDied","Data":"584b08c925936c423de4009e4a41b7523315abc63b4ab1cf3dc5e355752d86e5"}
Jul 15 15:55:01.673674 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:01.673600    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="584b08c925936c423de4009e4a41b7523315abc63b4ab1cf3dc5e355752d86e5"
Jul 15 15:55:01.704445 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:01.704412    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"]
Jul 15 15:55:01.706854 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:01.706828    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-5z7xk"]
Jul 15 15:55:01.768295 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:01.768261    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" path="/var/lib/kubelet/pods/10d0a192-5070-4e4a-abb7-bfc39a336676/volumes"
Jul 15 15:55:32.708098 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708051    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"]
Jul 15 15:55:32.710665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708442    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:55:32.710665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708454    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:55:32.710665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708477    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:55:32.710665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708482    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:55:32.710665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708548    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:55:32.710665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708557    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:55:32.710665 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.708563    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:55:32.711674 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.711657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"
Jul 15 15:55:32.714428 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.714407    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:55:32.721682 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.721658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"]
Jul 15 15:55:32.834362 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.834329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd351e5-5d6a-4fac-8f40-706ad71f5da9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9\" (UID: \"ebd351e5-5d6a-4fac-8f40-706ad71f5da9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"
Jul 15 15:55:32.935478 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.935452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd351e5-5d6a-4fac-8f40-706ad71f5da9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9\" (UID: \"ebd351e5-5d6a-4fac-8f40-706ad71f5da9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"
Jul 15 15:55:32.936115 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:32.936096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd351e5-5d6a-4fac-8f40-706ad71f5da9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9\" (UID: \"ebd351e5-5d6a-4fac-8f40-706ad71f5da9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"
Jul 15 15:55:33.022598 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:33.022527    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"
Jul 15 15:55:33.166013 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:33.165989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"]
Jul 15 15:55:33.167568 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:55:33.167539    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd351e5_5d6a_4fac_8f40_706ad71f5da9.slice/crio-514dd0fd245ee9e396c39452f6dffa61b9c307b9e0f94df840e0ea3f2eb36d08 WatchSource:0}: Error finding container 514dd0fd245ee9e396c39452f6dffa61b9c307b9e0f94df840e0ea3f2eb36d08: Status 404 returned error can't find the container with id 514dd0fd245ee9e396c39452f6dffa61b9c307b9e0f94df840e0ea3f2eb36d08
Jul 15 15:55:33.859980 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:33.859944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9" event={"ID":"ebd351e5-5d6a-4fac-8f40-706ad71f5da9","Type":"ContainerStarted","Data":"dc9bbafa2174b4ab8b5164616a7020d63d7a2f39146e15a4a98c98086286bf7c"}
Jul 15 15:55:33.860372 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:33.860026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9" event={"ID":"ebd351e5-5d6a-4fac-8f40-706ad71f5da9","Type":"ContainerStarted","Data":"514dd0fd245ee9e396c39452f6dffa61b9c307b9e0f94df840e0ea3f2eb36d08"}
Jul 15 15:55:33.878135 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:33.878086    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9" podStartSLOduration=1.878071569 podStartE2EDuration="1.878071569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:55:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:55:33.876928426 +0000 UTC m=+1858.678388240" watchObservedRunningTime="2026-07-15 15:55:33.878071569 +0000 UTC m=+1858.679531383"
Jul 15 15:55:34.867744 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:34.867710    2580 generic.go:356] "Generic (PLEG): container finished" podID="ebd351e5-5d6a-4fac-8f40-706ad71f5da9" containerID="dc9bbafa2174b4ab8b5164616a7020d63d7a2f39146e15a4a98c98086286bf7c" exitCode=0
Jul 15 15:55:34.868148 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:34.867793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9" event={"ID":"ebd351e5-5d6a-4fac-8f40-706ad71f5da9","Type":"ContainerDied","Data":"dc9bbafa2174b4ab8b5164616a7020d63d7a2f39146e15a4a98c98086286bf7c"}
Jul 15 15:55:36.004378 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:36.004353    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"
Jul 15 15:55:36.166171 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:36.166089    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd351e5-5d6a-4fac-8f40-706ad71f5da9-maas-api-service-ca\") pod \"ebd351e5-5d6a-4fac-8f40-706ad71f5da9\" (UID: \"ebd351e5-5d6a-4fac-8f40-706ad71f5da9\") "
Jul 15 15:55:36.166430 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:36.166411    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebd351e5-5d6a-4fac-8f40-706ad71f5da9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ebd351e5-5d6a-4fac-8f40-706ad71f5da9" (UID: "ebd351e5-5d6a-4fac-8f40-706ad71f5da9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:55:36.267402 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:36.267370    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebd351e5-5d6a-4fac-8f40-706ad71f5da9-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:55:36.881185 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:36.881158    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9"
Jul 15 15:55:36.881358 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:36.881200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9" event={"ID":"ebd351e5-5d6a-4fac-8f40-706ad71f5da9","Type":"ContainerDied","Data":"514dd0fd245ee9e396c39452f6dffa61b9c307b9e0f94df840e0ea3f2eb36d08"}
Jul 15 15:55:36.881358 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:36.881240    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="514dd0fd245ee9e396c39452f6dffa61b9c307b9e0f94df840e0ea3f2eb36d08"
Jul 15 15:55:43.181070 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.181032    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"]
Jul 15 15:55:43.181539 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.181262    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6" podUID="cf6ca770-eff0-4c1e-892b-bc5daf7a6259" containerName="maas-api" containerID="cri-o://dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069" gracePeriod=30
Jul 15 15:55:43.430816 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.430793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:55:43.535055 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.534968    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-openshift-service-ca\") pod \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") "
Jul 15 15:55:43.535221 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.535093    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls\") pod \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") "
Jul 15 15:55:43.535221 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.535187    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtlpn\" (UniqueName: \"kubernetes.io/projected/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-kube-api-access-jtlpn\") pod \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\" (UID: \"cf6ca770-eff0-4c1e-892b-bc5daf7a6259\") "
Jul 15 15:55:43.535358 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.535332    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf6ca770-eff0-4c1e-892b-bc5daf7a6259" (UID: "cf6ca770-eff0-4c1e-892b-bc5daf7a6259"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:55:43.535534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.535519    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:55:43.537083 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.537057    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf6ca770-eff0-4c1e-892b-bc5daf7a6259" (UID: "cf6ca770-eff0-4c1e-892b-bc5daf7a6259"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:55:43.537201 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.537132    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-kube-api-access-jtlpn" (OuterVolumeSpecName: "kube-api-access-jtlpn") pod "cf6ca770-eff0-4c1e-892b-bc5daf7a6259" (UID: "cf6ca770-eff0-4c1e-892b-bc5daf7a6259"). InnerVolumeSpecName "kube-api-access-jtlpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:55:43.636583 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.636555    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:55:43.636583 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.636581    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtlpn\" (UniqueName: \"kubernetes.io/projected/cf6ca770-eff0-4c1e-892b-bc5daf7a6259-kube-api-access-jtlpn\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:55:43.928402 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.928365    2580 generic.go:356] "Generic (PLEG): container finished" podID="cf6ca770-eff0-4c1e-892b-bc5daf7a6259" containerID="dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069" exitCode=0
Jul 15 15:55:43.928566 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.928411    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6" event={"ID":"cf6ca770-eff0-4c1e-892b-bc5daf7a6259","Type":"ContainerDied","Data":"dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069"}
Jul 15 15:55:43.928566 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.928426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"
Jul 15 15:55:43.928566 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.928479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6" event={"ID":"cf6ca770-eff0-4c1e-892b-bc5daf7a6259","Type":"ContainerDied","Data":"4f0faf767a0a25182fe242f6fbb39bf8ae337cb978cdccffc8e3b9d5a6fa7e71"}
Jul 15 15:55:43.928566 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.928507    2580 scope.go:117] "RemoveContainer" containerID="dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069"
Jul 15 15:55:43.938558 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.938540    2580 scope.go:117] "RemoveContainer" containerID="dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069"
Jul 15 15:55:43.938903 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:55:43.938882    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069\": container with ID starting with dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069 not found: ID does not exist" containerID="dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069"
Jul 15 15:55:43.938972 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.938910    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069"} err="failed to get container status \"dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069\": rpc error: code = NotFound desc = could not find container \"dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069\": container with ID starting with dde918c9f9189f1acc2b5fd33cc60af90348dfb7863dab293d14f60195e2b069 not found: ID does not exist"
Jul 15 15:55:43.949355 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.949332    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"]
Jul 15 15:55:43.951658 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:43.951638    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d69cf00-59c8b78b9f-9mvc6"]
Jul 15 15:55:45.768880 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:45.768847    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf6ca770-eff0-4c1e-892b-bc5daf7a6259" path="/var/lib/kubelet/pods/cf6ca770-eff0-4c1e-892b-bc5daf7a6259/volumes"
Jul 15 15:55:46.075133 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:55:46.075107    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:57:22.283344 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:22.283314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:57:31.881126 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:31.880926    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:57:35.895298 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:35.895268    2580 scope.go:117] "RemoveContainer" containerID="5b5f730dcc491effe54a1468199fb1e72c47c4f760527c477ad4335f547d23cd"
Jul 15 15:57:39.068053 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068020    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"]
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068406    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068417    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d0a192-5070-4e4a-abb7-bfc39a336676" containerName="revoke-keys"
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068431    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6ca770-eff0-4c1e-892b-bc5daf7a6259" containerName="maas-api"
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068436    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6ca770-eff0-4c1e-892b-bc5daf7a6259" containerName="maas-api"
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068457    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd351e5-5d6a-4fac-8f40-706ad71f5da9" containerName="revoke-keys"
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068462    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd351e5-5d6a-4fac-8f40-706ad71f5da9" containerName="revoke-keys"
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068520    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebd351e5-5d6a-4fac-8f40-706ad71f5da9" containerName="revoke-keys"
Jul 15 15:57:39.068534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.068529    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6ca770-eff0-4c1e-892b-bc5daf7a6259" containerName="maas-api"
Jul 15 15:57:39.071853 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.071832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.074969 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.074949    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 15:57:39.075092 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.075005    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 15:57:39.075736 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.075708    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 15:57:39.075883 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.075796    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7258970a-serving-cert\""
Jul 15 15:57:39.083132 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.083107    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"]
Jul 15 15:57:39.170200 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.170175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.170360 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.170206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjzkn\" (UniqueName: \"kubernetes.io/projected/e44d759a-561a-4866-a3ad-e0d4ee5bd059-kube-api-access-qjzkn\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.170360 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.170243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44d759a-561a-4866-a3ad-e0d4ee5bd059-openshift-service-ca\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.270958 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.270926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjzkn\" (UniqueName: \"kubernetes.io/projected/e44d759a-561a-4866-a3ad-e0d4ee5bd059-kube-api-access-qjzkn\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.271117 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.270975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44d759a-561a-4866-a3ad-e0d4ee5bd059-openshift-service-ca\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.271117 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.271056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.271190 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:57:39.271155    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-serving-cert: secret "maas-api-e2e-mt-7258970a-serving-cert" not found
Jul 15 15:57:39.271241 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:57:39.271230    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls podName:e44d759a-561a-4866-a3ad-e0d4ee5bd059 nodeName:}" failed. No retries permitted until 2026-07-15 15:57:39.771209846 +0000 UTC m=+1984.572669640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls") pod "maas-api-e2e-mt-7258970a-599867b658-8tpn4" (UID: "e44d759a-561a-4866-a3ad-e0d4ee5bd059") : secret "maas-api-e2e-mt-7258970a-serving-cert" not found
Jul 15 15:57:39.271727 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.271708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44d759a-561a-4866-a3ad-e0d4ee5bd059-openshift-service-ca\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.281052 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.281030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjzkn\" (UniqueName: \"kubernetes.io/projected/e44d759a-561a-4866-a3ad-e0d4ee5bd059-kube-api-access-qjzkn\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.774842 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.774810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.777270 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.777245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls\") pod \"maas-api-e2e-mt-7258970a-599867b658-8tpn4\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:39.983034 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:39.982992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:40.121408 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:40.121380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"]
Jul 15 15:57:40.122508 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:57:40.122477    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44d759a_561a_4866_a3ad_e0d4ee5bd059.slice/crio-db72c09763e006351962f4a3dbff574dd084bcd46547e43f62053bad520c4d81 WatchSource:0}: Error finding container db72c09763e006351962f4a3dbff574dd084bcd46547e43f62053bad520c4d81: Status 404 returned error can't find the container with id db72c09763e006351962f4a3dbff574dd084bcd46547e43f62053bad520c4d81
Jul 15 15:57:40.123903 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:40.123883    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 15:57:40.629960 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:40.629934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4" event={"ID":"e44d759a-561a-4866-a3ad-e0d4ee5bd059","Type":"ContainerStarted","Data":"db72c09763e006351962f4a3dbff574dd084bcd46547e43f62053bad520c4d81"}
Jul 15 15:57:41.637388 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:41.637354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4" event={"ID":"e44d759a-561a-4866-a3ad-e0d4ee5bd059","Type":"ContainerStarted","Data":"7dd458c9e957d5400cec70982e2881ee4cc59b3d0d6c9ce6ddf6782c9bc7e5d1"}
Jul 15 15:57:41.637791 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:41.637551    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:41.656101 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:41.656012    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4" podStartSLOduration=2.181739131 podStartE2EDuration="2.655996909s" totalImagesPullingTime="474.257778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:57:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:57:41.654233391 +0000 UTC m=+1986.455693206" watchObservedRunningTime="2026-07-15 15:57:41.655996909 +0000 UTC m=+1986.457456725"
Jul 15 15:57:43.882167 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:43.882133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:57:47.649358 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:47.649329    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:57:51.141822 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.141789    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"]
Jul 15 15:57:51.145569 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.145545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"
Jul 15 15:57:51.148240 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.148215    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:57:51.154061 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.154038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"]
Jul 15 15:57:51.276713 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.276679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1659b-0c51-4dc1-a33d-55c13047134c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7258970a-8dhkj\" (UID: \"9ce1659b-0c51-4dc1-a33d-55c13047134c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"
Jul 15 15:57:51.378083 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.378054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1659b-0c51-4dc1-a33d-55c13047134c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7258970a-8dhkj\" (UID: \"9ce1659b-0c51-4dc1-a33d-55c13047134c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"
Jul 15 15:57:51.378644 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.378622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1659b-0c51-4dc1-a33d-55c13047134c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7258970a-8dhkj\" (UID: \"9ce1659b-0c51-4dc1-a33d-55c13047134c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"
Jul 15 15:57:51.459357 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.459283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"
Jul 15 15:57:51.596940 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.596916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"]
Jul 15 15:57:51.598418 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:57:51.598381    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce1659b_0c51_4dc1_a33d_55c13047134c.slice/crio-2904ec139fc9a428ec9243ea33597ddf54fa7abf87b0557e3a493f18858b61a5 WatchSource:0}: Error finding container 2904ec139fc9a428ec9243ea33597ddf54fa7abf87b0557e3a493f18858b61a5: Status 404 returned error can't find the container with id 2904ec139fc9a428ec9243ea33597ddf54fa7abf87b0557e3a493f18858b61a5
Jul 15 15:57:51.697085 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:51.697056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj" event={"ID":"9ce1659b-0c51-4dc1-a33d-55c13047134c","Type":"ContainerStarted","Data":"2904ec139fc9a428ec9243ea33597ddf54fa7abf87b0557e3a493f18858b61a5"}
Jul 15 15:57:52.704613 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:52.704581    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ce1659b-0c51-4dc1-a33d-55c13047134c" containerID="97ff663c13b99c0128d596a1bc60d3068b5eaa5818e2515f1250a2a703287cda" exitCode=0
Jul 15 15:57:52.705013 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:52.704664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj" event={"ID":"9ce1659b-0c51-4dc1-a33d-55c13047134c","Type":"ContainerDied","Data":"97ff663c13b99c0128d596a1bc60d3068b5eaa5818e2515f1250a2a703287cda"}
Jul 15 15:57:53.842588 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:53.842566    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"
Jul 15 15:57:54.011090 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:54.011007    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1659b-0c51-4dc1-a33d-55c13047134c-maas-api-service-ca\") pod \"9ce1659b-0c51-4dc1-a33d-55c13047134c\" (UID: \"9ce1659b-0c51-4dc1-a33d-55c13047134c\") "
Jul 15 15:57:54.011354 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:54.011330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce1659b-0c51-4dc1-a33d-55c13047134c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ce1659b-0c51-4dc1-a33d-55c13047134c" (UID: "9ce1659b-0c51-4dc1-a33d-55c13047134c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:57:54.112534 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:54.112504    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1659b-0c51-4dc1-a33d-55c13047134c-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:57:54.717839 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:54.717804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj" event={"ID":"9ce1659b-0c51-4dc1-a33d-55c13047134c","Type":"ContainerDied","Data":"2904ec139fc9a428ec9243ea33597ddf54fa7abf87b0557e3a493f18858b61a5"}
Jul 15 15:57:54.717988 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:54.717859    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2904ec139fc9a428ec9243ea33597ddf54fa7abf87b0557e3a493f18858b61a5"
Jul 15 15:57:54.717988 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:57:54.717882    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7258970a-8dhkj"
Jul 15 15:58:01.624003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.623214    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"]
Jul 15 15:58:01.624003 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.623517    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4" podUID="e44d759a-561a-4866-a3ad-e0d4ee5bd059" containerName="maas-api" containerID="cri-o://7dd458c9e957d5400cec70982e2881ee4cc59b3d0d6c9ce6ddf6782c9bc7e5d1" gracePeriod=30
Jul 15 15:58:01.763983 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.763944    2580 generic.go:356] "Generic (PLEG): container finished" podID="e44d759a-561a-4866-a3ad-e0d4ee5bd059" containerID="7dd458c9e957d5400cec70982e2881ee4cc59b3d0d6c9ce6ddf6782c9bc7e5d1" exitCode=0
Jul 15 15:58:01.768308 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.768271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4" event={"ID":"e44d759a-561a-4866-a3ad-e0d4ee5bd059","Type":"ContainerDied","Data":"7dd458c9e957d5400cec70982e2881ee4cc59b3d0d6c9ce6ddf6782c9bc7e5d1"}
Jul 15 15:58:01.882971 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.882908    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:58:01.980719 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.980687    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls\") pod \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") "
Jul 15 15:58:01.980898 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.980787    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44d759a-561a-4866-a3ad-e0d4ee5bd059-openshift-service-ca\") pod \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") "
Jul 15 15:58:01.980898 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.980811    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjzkn\" (UniqueName: \"kubernetes.io/projected/e44d759a-561a-4866-a3ad-e0d4ee5bd059-kube-api-access-qjzkn\") pod \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\" (UID: \"e44d759a-561a-4866-a3ad-e0d4ee5bd059\") "
Jul 15 15:58:01.981183 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.981155    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44d759a-561a-4866-a3ad-e0d4ee5bd059-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e44d759a-561a-4866-a3ad-e0d4ee5bd059" (UID: "e44d759a-561a-4866-a3ad-e0d4ee5bd059"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:58:01.982794 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.982771    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e44d759a-561a-4866-a3ad-e0d4ee5bd059" (UID: "e44d759a-561a-4866-a3ad-e0d4ee5bd059"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:58:01.982904 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:01.982857    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44d759a-561a-4866-a3ad-e0d4ee5bd059-kube-api-access-qjzkn" (OuterVolumeSpecName: "kube-api-access-qjzkn") pod "e44d759a-561a-4866-a3ad-e0d4ee5bd059" (UID: "e44d759a-561a-4866-a3ad-e0d4ee5bd059"). InnerVolumeSpecName "kube-api-access-qjzkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:58:02.081826 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.081796    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44d759a-561a-4866-a3ad-e0d4ee5bd059-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:58:02.081826 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.081822    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjzkn\" (UniqueName: \"kubernetes.io/projected/e44d759a-561a-4866-a3ad-e0d4ee5bd059-kube-api-access-qjzkn\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:58:02.081826 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.081832    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44d759a-561a-4866-a3ad-e0d4ee5bd059-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:58:02.771593 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.771562    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"
Jul 15 15:58:02.772030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.771607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4" event={"ID":"e44d759a-561a-4866-a3ad-e0d4ee5bd059","Type":"ContainerDied","Data":"db72c09763e006351962f4a3dbff574dd084bcd46547e43f62053bad520c4d81"}
Jul 15 15:58:02.772030 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.771666    2580 scope.go:117] "RemoveContainer" containerID="7dd458c9e957d5400cec70982e2881ee4cc59b3d0d6c9ce6ddf6782c9bc7e5d1"
Jul 15 15:58:02.794809 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.794786    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"]
Jul 15 15:58:02.798792 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:02.798771    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7258970a-599867b658-8tpn4"]
Jul 15 15:58:03.768095 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:03.768057    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e44d759a-561a-4866-a3ad-e0d4ee5bd059" path="/var/lib/kubelet/pods/e44d759a-561a-4866-a3ad-e0d4ee5bd059/volumes"
Jul 15 15:58:04.272405 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:04.272373    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:58:35.919588 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:35.919554    2580 scope.go:117] "RemoveContainer" containerID="e3a169e2a69ce59ced3bc22db9fe452cbf7441e85c144ddcbb31231b044bb2d1"
Jul 15 15:58:40.429335 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.429297    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"]
Jul 15 15:58:40.429815 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.429794    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e44d759a-561a-4866-a3ad-e0d4ee5bd059" containerName="maas-api"
Jul 15 15:58:40.429872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.429819    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e44d759a-561a-4866-a3ad-e0d4ee5bd059" containerName="maas-api"
Jul 15 15:58:40.429872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.429855    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce1659b-0c51-4dc1-a33d-55c13047134c" containerName="revoke-keys"
Jul 15 15:58:40.429872 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.429866    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce1659b-0c51-4dc1-a33d-55c13047134c" containerName="revoke-keys"
Jul 15 15:58:40.429991 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.429977    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e44d759a-561a-4866-a3ad-e0d4ee5bd059" containerName="maas-api"
Jul 15 15:58:40.430026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.429998    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce1659b-0c51-4dc1-a33d-55c13047134c" containerName="revoke-keys"
Jul 15 15:58:40.433171 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.433150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.436106 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.436085    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 15:58:40.436380 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.436367    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 15:58:40.437292 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.437275    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fb1842df-serving-cert\""
Jul 15 15:58:40.437341 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.437283    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 15:58:40.452616 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.452595    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"]
Jul 15 15:58:40.530904 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.530876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-maas-api-tls\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.531035 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.530995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-openshift-service-ca\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.531082 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.531060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrtr5\" (UniqueName: \"kubernetes.io/projected/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-kube-api-access-lrtr5\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.631829 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.631801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrtr5\" (UniqueName: \"kubernetes.io/projected/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-kube-api-access-lrtr5\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.631968 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.631835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-maas-api-tls\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.631968 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.631918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-openshift-service-ca\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.632616 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.632597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-openshift-service-ca\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.634273 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.634252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-maas-api-tls\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.641716 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.641694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrtr5\" (UniqueName: \"kubernetes.io/projected/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-kube-api-access-lrtr5\") pod \"maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.743464 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.743381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:40.881766 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.881719    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"]
Jul 15 15:58:40.882780 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:58:40.882721    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e3e2fd_50ba_4b55_8fd3_95d44a63f7c9.slice/crio-a9a326500122f6e02a7f114836cc3c03867476442b6c7b79bf9b07e8781c2f10 WatchSource:0}: Error finding container a9a326500122f6e02a7f114836cc3c03867476442b6c7b79bf9b07e8781c2f10: Status 404 returned error can't find the container with id a9a326500122f6e02a7f114836cc3c03867476442b6c7b79bf9b07e8781c2f10
Jul 15 15:58:40.998216 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:40.998119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt" event={"ID":"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9","Type":"ContainerStarted","Data":"a9a326500122f6e02a7f114836cc3c03867476442b6c7b79bf9b07e8781c2f10"}
Jul 15 15:58:42.006011 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:42.005962    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt" event={"ID":"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9","Type":"ContainerStarted","Data":"39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b"}
Jul 15 15:58:42.006469 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:42.006106    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:42.025992 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:42.025949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt" podStartSLOduration=1.6676799450000002 podStartE2EDuration="2.025933932s" totalImagesPullingTime="358.253987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:58:42.023608411 +0000 UTC m=+2046.825068256" watchObservedRunningTime="2026-07-15 15:58:42.025933932 +0000 UTC m=+2046.827393758"
Jul 15 15:58:46.374501 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:46.374469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:58:48.017789 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:48.017735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:58:49.771279 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.771248    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"]
Jul 15 15:58:49.775087 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.775069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.778439 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.778417    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fb1842df\"/\"e2e-lifecycle-model-fb1842df-kserve-self-signed-certs\""
Jul 15 15:58:49.779466 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.779445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fb1842df\"/\"kube-root-ca.crt\""
Jul 15 15:58:49.779582 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.779550    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fb1842df\"/\"default-dockercfg-2qbzg\""
Jul 15 15:58:49.779687 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.779653    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fb1842df\"/\"openshift-service-ca.crt\""
Jul 15 15:58:49.786502 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.786481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"]
Jul 15 15:58:49.815608 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.815571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tmp-dir\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.815608 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.815611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t77t\" (UniqueName: \"kubernetes.io/projected/8824afe6-c141-40f8-8ffd-c591d31c6fdb-kube-api-access-9t77t\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.815835 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.815718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-dshm\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.815835 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.815772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-home\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.815835 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.815803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-model-cache\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.815835 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.815827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tls-certs\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916194 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-home\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916348 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-model-cache\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916348 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tls-certs\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916348 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tmp-dir\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916348 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t77t\" (UniqueName: \"kubernetes.io/projected/8824afe6-c141-40f8-8ffd-c591d31c6fdb-kube-api-access-9t77t\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916584 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-dshm\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916765 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tmp-dir\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916849 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-model-cache\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.916849 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.916790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-home\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.918578 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.918560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-dshm\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.918743 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.918728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tls-certs\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:49.925270 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:49.925249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t77t\" (UniqueName: \"kubernetes.io/projected/8824afe6-c141-40f8-8ffd-c591d31c6fdb-kube-api-access-9t77t\") pod \"e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") " pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:50.086578 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:50.086546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:58:50.225026 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:50.225002    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"]
Jul 15 15:58:50.226402 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:58:50.226376    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8824afe6_c141_40f8_8ffd_c591d31c6fdb.slice/crio-f15d23f56c5ab8030129cc543a38003cc999ba9fa041252badd2c68494c5c955 WatchSource:0}: Error finding container f15d23f56c5ab8030129cc543a38003cc999ba9fa041252badd2c68494c5c955: Status 404 returned error can't find the container with id f15d23f56c5ab8030129cc543a38003cc999ba9fa041252badd2c68494c5c955
Jul 15 15:58:51.060403 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:51.060366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl" event={"ID":"8824afe6-c141-40f8-8ffd-c591d31c6fdb","Type":"ContainerStarted","Data":"1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587"}
Jul 15 15:58:51.060787 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:51.060421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl" event={"ID":"8824afe6-c141-40f8-8ffd-c591d31c6fdb","Type":"ContainerStarted","Data":"f15d23f56c5ab8030129cc543a38003cc999ba9fa041252badd2c68494c5c955"}
Jul 15 15:58:51.081056 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:51.081012    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl" podStartSLOduration=2.080998791 podStartE2EDuration="2.080998791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 15:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 15:58:51.078418332 +0000 UTC m=+2055.879878147" watchObservedRunningTime="2026-07-15 15:58:51.080998791 +0000 UTC m=+2055.882458634"
Jul 15 15:58:55.462266 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:58:55.462232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:59:00.086735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:00.086647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:59:00.086735 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:00.086697    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:59:00.099359 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:00.099337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:59:00.123893 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:00.123868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:59:04.221163 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:04.221123    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:59:07.707854 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:07.707816    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"]
Jul 15 15:59:07.711430 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:07.711407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"
Jul 15 15:59:07.714118 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:07.714098    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 15:59:07.720577 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:07.720440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"]
Jul 15 15:59:07.780193 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:07.780165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12129e48-ab0f-443b-8ef6-0ac401ad6c57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fb1842df-j474n\" (UID: \"12129e48-ab0f-443b-8ef6-0ac401ad6c57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"
Jul 15 15:59:07.881439 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:07.881400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12129e48-ab0f-443b-8ef6-0ac401ad6c57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fb1842df-j474n\" (UID: \"12129e48-ab0f-443b-8ef6-0ac401ad6c57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"
Jul 15 15:59:07.882191 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:07.882170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12129e48-ab0f-443b-8ef6-0ac401ad6c57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fb1842df-j474n\" (UID: \"12129e48-ab0f-443b-8ef6-0ac401ad6c57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"
Jul 15 15:59:08.025705 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:08.025626    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"
Jul 15 15:59:08.160865 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:08.160834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"]
Jul 15 15:59:08.161979 ip-10-0-135-125 kubenswrapper[2580]: W0715 15:59:08.161950    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12129e48_ab0f_443b_8ef6_0ac401ad6c57.slice/crio-916ac30b9d613656f1070c1af8b2a34815d7b5777afbd395940b81308b88481b WatchSource:0}: Error finding container 916ac30b9d613656f1070c1af8b2a34815d7b5777afbd395940b81308b88481b: Status 404 returned error can't find the container with id 916ac30b9d613656f1070c1af8b2a34815d7b5777afbd395940b81308b88481b
Jul 15 15:59:09.171440 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:09.171404    2580 generic.go:356] "Generic (PLEG): container finished" podID="12129e48-ab0f-443b-8ef6-0ac401ad6c57" containerID="e114c9840395ae32c131709062959f07405eff253c2f9583b20f94cc739f9792" exitCode=0
Jul 15 15:59:09.171840 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:09.171488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n" event={"ID":"12129e48-ab0f-443b-8ef6-0ac401ad6c57","Type":"ContainerDied","Data":"e114c9840395ae32c131709062959f07405eff253c2f9583b20f94cc739f9792"}
Jul 15 15:59:09.171840 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:09.171540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n" event={"ID":"12129e48-ab0f-443b-8ef6-0ac401ad6c57","Type":"ContainerStarted","Data":"916ac30b9d613656f1070c1af8b2a34815d7b5777afbd395940b81308b88481b"}
Jul 15 15:59:10.309628 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:10.309599    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"
Jul 15 15:59:10.410046 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:10.409999    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12129e48-ab0f-443b-8ef6-0ac401ad6c57-maas-api-service-ca\") pod \"12129e48-ab0f-443b-8ef6-0ac401ad6c57\" (UID: \"12129e48-ab0f-443b-8ef6-0ac401ad6c57\") "
Jul 15 15:59:10.410344 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:10.410320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12129e48-ab0f-443b-8ef6-0ac401ad6c57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12129e48-ab0f-443b-8ef6-0ac401ad6c57" (UID: "12129e48-ab0f-443b-8ef6-0ac401ad6c57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:59:10.511281 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:10.511201    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12129e48-ab0f-443b-8ef6-0ac401ad6c57-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:11.183649 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:11.183623    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n"
Jul 15 15:59:11.183842 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:11.183671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fb1842df-j474n" event={"ID":"12129e48-ab0f-443b-8ef6-0ac401ad6c57","Type":"ContainerDied","Data":"916ac30b9d613656f1070c1af8b2a34815d7b5777afbd395940b81308b88481b"}
Jul 15 15:59:11.183842 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:11.183719    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="916ac30b9d613656f1070c1af8b2a34815d7b5777afbd395940b81308b88481b"
Jul 15 15:59:18.288535 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.288497    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"]
Jul 15 15:59:18.289310 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.289269    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt" podUID="b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" containerName="maas-api" containerID="cri-o://39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b" gracePeriod=30
Jul 15 15:59:18.543967 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.543908    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:59:18.686459 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.686419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-openshift-service-ca\") pod \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") "
Jul 15 15:59:18.686650 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.686537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-maas-api-tls\") pod \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") "
Jul 15 15:59:18.686650 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.686566    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrtr5\" (UniqueName: \"kubernetes.io/projected/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-kube-api-access-lrtr5\") pod \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\" (UID: \"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9\") "
Jul 15 15:59:18.686866 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.686842    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" (UID: "b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 15:59:18.688606 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.688575    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" (UID: "b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:59:18.688698 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.688645    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-kube-api-access-lrtr5" (OuterVolumeSpecName: "kube-api-access-lrtr5") pod "b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" (UID: "b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9"). InnerVolumeSpecName "kube-api-access-lrtr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:59:18.787438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.787391    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:18.787438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.787434    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrtr5\" (UniqueName: \"kubernetes.io/projected/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-kube-api-access-lrtr5\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:18.787438 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:18.787445    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:19.234285 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.234251    2580 generic.go:356] "Generic (PLEG): container finished" podID="b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" containerID="39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b" exitCode=0
Jul 15 15:59:19.234444 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.234307    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"
Jul 15 15:59:19.234444 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.234338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt" event={"ID":"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9","Type":"ContainerDied","Data":"39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b"}
Jul 15 15:59:19.234444 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.234412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt" event={"ID":"b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9","Type":"ContainerDied","Data":"a9a326500122f6e02a7f114836cc3c03867476442b6c7b79bf9b07e8781c2f10"}
Jul 15 15:59:19.234444 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.234435    2580 scope.go:117] "RemoveContainer" containerID="39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b"
Jul 15 15:59:19.244585 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.244566    2580 scope.go:117] "RemoveContainer" containerID="39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b"
Jul 15 15:59:19.244852 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:59:19.244834    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b\": container with ID starting with 39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b not found: ID does not exist" containerID="39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b"
Jul 15 15:59:19.244929 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.244860    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b"} err="failed to get container status \"39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b\": rpc error: code = NotFound desc = could not find container \"39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b\": container with ID starting with 39810237a7a1bdf9db50606921dbfcc919327ae8aa316d4879433040541ffb1b not found: ID does not exist"
Jul 15 15:59:19.262288 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.262257    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"]
Jul 15 15:59:19.265514 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.265495    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb1842df-c44d6bb6d-swskt"]
Jul 15 15:59:19.767973 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:19.767939    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" path="/var/lib/kubelet/pods/b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9/volumes"
Jul 15 15:59:22.267831 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:22.267799    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 15:59:27.789403 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:27.789373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"]
Jul 15 15:59:35.856623 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:35.856587    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:59:35.864355 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:35.864336    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 15:59:35.945952 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:35.945925    2580 scope.go:117] "RemoveContainer" containerID="09a55aa9f26a746acacf60c463b98ad3c23bcc0d3f901d00256823853ae693d0"
Jul 15 15:59:42.812238 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:42.812170    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl" podUID="8824afe6-c141-40f8-8ffd-c591d31c6fdb" containerName="main" containerID="cri-o://1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587" gracePeriod=45
Jul 15 15:59:43.082854 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.082827    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:59:43.207153 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207119    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t77t\" (UniqueName: \"kubernetes.io/projected/8824afe6-c141-40f8-8ffd-c591d31c6fdb-kube-api-access-9t77t\") pod \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") "
Jul 15 15:59:43.207331 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207187    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-dshm\") pod \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") "
Jul 15 15:59:43.207411 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207348    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-model-cache\") pod \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") "
Jul 15 15:59:43.207470 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207421    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tmp-dir\") pod \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") "
Jul 15 15:59:43.207535 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207476    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-home\") pod \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") "
Jul 15 15:59:43.207535 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tls-certs\") pod \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\" (UID: \"8824afe6-c141-40f8-8ffd-c591d31c6fdb\") "
Jul 15 15:59:43.207633 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207584    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8824afe6-c141-40f8-8ffd-c591d31c6fdb" (UID: "8824afe6-c141-40f8-8ffd-c591d31c6fdb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:59:43.207633 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207605    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-model-cache" (OuterVolumeSpecName: "model-cache") pod "8824afe6-c141-40f8-8ffd-c591d31c6fdb" (UID: "8824afe6-c141-40f8-8ffd-c591d31c6fdb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:59:43.207835 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207790    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-home" (OuterVolumeSpecName: "home") pod "8824afe6-c141-40f8-8ffd-c591d31c6fdb" (UID: "8824afe6-c141-40f8-8ffd-c591d31c6fdb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:59:43.207915 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207896    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:43.207979 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207939    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:43.207979 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.207952    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:43.209451 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.209427    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-dshm" (OuterVolumeSpecName: "dshm") pod "8824afe6-c141-40f8-8ffd-c591d31c6fdb" (UID: "8824afe6-c141-40f8-8ffd-c591d31c6fdb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 15:59:43.209568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.209499    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8824afe6-c141-40f8-8ffd-c591d31c6fdb-kube-api-access-9t77t" (OuterVolumeSpecName: "kube-api-access-9t77t") pod "8824afe6-c141-40f8-8ffd-c591d31c6fdb" (UID: "8824afe6-c141-40f8-8ffd-c591d31c6fdb"). InnerVolumeSpecName "kube-api-access-9t77t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 15:59:43.209568 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.209520    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8824afe6-c141-40f8-8ffd-c591d31c6fdb" (UID: "8824afe6-c141-40f8-8ffd-c591d31c6fdb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 15:59:43.308836 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.308811    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t77t\" (UniqueName: \"kubernetes.io/projected/8824afe6-c141-40f8-8ffd-c591d31c6fdb-kube-api-access-9t77t\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:43.308836 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.308833    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8824afe6-c141-40f8-8ffd-c591d31c6fdb-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:43.308966 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.308843    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8824afe6-c141-40f8-8ffd-c591d31c6fdb-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 15:59:43.391707 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.391677    2580 generic.go:356] "Generic (PLEG): container finished" podID="8824afe6-c141-40f8-8ffd-c591d31c6fdb" containerID="1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587" exitCode=0
Jul 15 15:59:43.391896 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.391723    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"
Jul 15 15:59:43.391896 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.391735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl" event={"ID":"8824afe6-c141-40f8-8ffd-c591d31c6fdb","Type":"ContainerDied","Data":"1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587"}
Jul 15 15:59:43.391896 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.391799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl" event={"ID":"8824afe6-c141-40f8-8ffd-c591d31c6fdb","Type":"ContainerDied","Data":"f15d23f56c5ab8030129cc543a38003cc999ba9fa041252badd2c68494c5c955"}
Jul 15 15:59:43.391896 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.391822    2580 scope.go:117] "RemoveContainer" containerID="1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587"
Jul 15 15:59:43.401696 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.401677    2580 scope.go:117] "RemoveContainer" containerID="1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587"
Jul 15 15:59:43.401969 ip-10-0-135-125 kubenswrapper[2580]: E0715 15:59:43.401951    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587\": container with ID starting with 1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587 not found: ID does not exist" containerID="1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587"
Jul 15 15:59:43.402015 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.401978    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587"} err="failed to get container status \"1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587\": rpc error: code = NotFound desc = could not find container \"1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587\": container with ID starting with 1d442ef7803e6b2142c1daa2ba6cfc6e7b9eae051731b4b7a1da74a20b1dd587 not found: ID does not exist"
Jul 15 15:59:43.417458 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.417434    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"]
Jul 15 15:59:43.419969 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.419950    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fb1842df/e2e-lifecycle-model-fb1842df-kserve-cd8d5955-hpmwl"]
Jul 15 15:59:43.767477 ip-10-0-135-125 kubenswrapper[2580]: I0715 15:59:43.767397    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8824afe6-c141-40f8-8ffd-c591d31c6fdb" path="/var/lib/kubelet/pods/8824afe6-c141-40f8-8ffd-c591d31c6fdb/volumes"
Jul 15 16:00:00.159575 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.159539    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588"]
Jul 15 16:00:00.160040 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.159981    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" containerName="maas-api"
Jul 15 16:00:00.160040 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.159996    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" containerName="maas-api"
Jul 15 16:00:00.160040 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.160004    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12129e48-ab0f-443b-8ef6-0ac401ad6c57" containerName="revoke-keys"
Jul 15 16:00:00.160040 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.160010    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="12129e48-ab0f-443b-8ef6-0ac401ad6c57" containerName="revoke-keys"
Jul 15 16:00:00.160040 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.160030    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8824afe6-c141-40f8-8ffd-c591d31c6fdb" containerName="main"
Jul 15 16:00:00.160040 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.160037    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8824afe6-c141-40f8-8ffd-c591d31c6fdb" containerName="main"
Jul 15 16:00:00.160252 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.160115    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e3e2fd-50ba-4b55-8fd3-95d44a63f7c9" containerName="maas-api"
Jul 15 16:00:00.160252 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.160126    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8824afe6-c141-40f8-8ffd-c591d31c6fdb" containerName="main"
Jul 15 16:00:00.160252 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.160138    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="12129e48-ab0f-443b-8ef6-0ac401ad6c57" containerName="revoke-keys"
Jul 15 16:00:00.164814 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.164796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588"
Jul 15 16:00:00.167915 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.167891    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 16:00:00.175196 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.175171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588"]
Jul 15 16:00:00.176146 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.176128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588"
Jul 15 16:00:00.320715 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.320692    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588"]
Jul 15 16:00:00.322543 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:00:00.322514    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4360de3e_6110_48a2_9763_858e3987a591.slice/crio-a74cc3ca2a01aebb6589ee9335176acdb62d6585e8623aced1b5b0d4f902fbcc WatchSource:0}: Error finding container a74cc3ca2a01aebb6589ee9335176acdb62d6585e8623aced1b5b0d4f902fbcc: Status 404 returned error can't find the container with id a74cc3ca2a01aebb6589ee9335176acdb62d6585e8623aced1b5b0d4f902fbcc
Jul 15 16:00:00.494166 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:00.494077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588" event={"ID":"4360de3e-6110-48a2-9763-858e3987a591","Type":"ContainerStarted","Data":"a74cc3ca2a01aebb6589ee9335176acdb62d6585e8623aced1b5b0d4f902fbcc"}
Jul 15 16:00:01.501926 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:01.501828    2580 generic.go:356] "Generic (PLEG): container finished" podID="4360de3e-6110-48a2-9763-858e3987a591" containerID="7f4d684a77e921be0692e6563078d0284fb9b1d464a441105c7eecb21706c26f" exitCode=0
Jul 15 16:00:01.501926 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:01.501905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588" event={"ID":"4360de3e-6110-48a2-9763-858e3987a591","Type":"ContainerDied","Data":"7f4d684a77e921be0692e6563078d0284fb9b1d464a441105c7eecb21706c26f"}
Jul 15 16:00:02.643534 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:02.643505    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588"
Jul 15 16:00:03.515604 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:03.515572    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588"
Jul 15 16:00:03.515805 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:03.515614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735520-57588" event={"ID":"4360de3e-6110-48a2-9763-858e3987a591","Type":"ContainerDied","Data":"a74cc3ca2a01aebb6589ee9335176acdb62d6585e8623aced1b5b0d4f902fbcc"}
Jul 15 16:00:03.515805 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:03.515660    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a74cc3ca2a01aebb6589ee9335176acdb62d6585e8623aced1b5b0d4f902fbcc"
Jul 15 16:00:29.474829 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:29.474792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:00:42.777306 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:00:42.777272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:01:24.115276 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.115244    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"]
Jul 15 16:01:24.115734 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.115634    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4360de3e-6110-48a2-9763-858e3987a591" containerName="cleanup"
Jul 15 16:01:24.115734 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.115645    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4360de3e-6110-48a2-9763-858e3987a591" containerName="cleanup"
Jul 15 16:01:24.115734 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.115723    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4360de3e-6110-48a2-9763-858e3987a591" containerName="cleanup"
Jul 15 16:01:24.118767 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.118737    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.122006 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.121986    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a920e4-serving-cert\""
Jul 15 16:01:24.122989 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.122970    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 16:01:24.123496 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.123483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:01:24.123978 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.123960    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 16:01:24.149238 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.149213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"]
Jul 15 16:01:24.157087 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.157063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.157205 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.157108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfp72\" (UniqueName: \"kubernetes.io/projected/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-kube-api-access-mfp72\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.157205 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.157191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-maas-api-tls\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.258157 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.258125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.258333 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.258167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfp72\" (UniqueName: \"kubernetes.io/projected/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-kube-api-access-mfp72\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.258333 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.258204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-maas-api-tls\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.258819 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.258798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.260721 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.260694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-maas-api-tls\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.277401 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.277374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfp72\" (UniqueName: \"kubernetes.io/projected/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-kube-api-access-mfp72\") pod \"maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.429746 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.429664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:24.567098 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.567073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"]
Jul 15 16:01:24.568493 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:01:24.568461    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod211dc51e_a8ec_4fbd_a8b7_a6ceb0a2d69d.slice/crio-cd9bd4e4f47aff4700afdf09ec1fa26338e9dbb994664de6320cc97cd2d31709 WatchSource:0}: Error finding container cd9bd4e4f47aff4700afdf09ec1fa26338e9dbb994664de6320cc97cd2d31709: Status 404 returned error can't find the container with id cd9bd4e4f47aff4700afdf09ec1fa26338e9dbb994664de6320cc97cd2d31709
Jul 15 16:01:24.995594 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:24.995563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj" event={"ID":"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d","Type":"ContainerStarted","Data":"cd9bd4e4f47aff4700afdf09ec1fa26338e9dbb994664de6320cc97cd2d31709"}
Jul 15 16:01:28.022471 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:28.022365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj" event={"ID":"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d","Type":"ContainerStarted","Data":"5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6"}
Jul 15 16:01:28.022889 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:28.022508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:28.042657 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:28.042612    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj" podStartSLOduration=1.007100829 podStartE2EDuration="4.042597945s" totalImagesPullingTime="3.035497116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:01:28.041275584 +0000 UTC m=+2212.842735406" watchObservedRunningTime="2026-07-15 16:01:28.042597945 +0000 UTC m=+2212.844057758"
Jul 15 16:01:32.144987 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.144957    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"]
Jul 15 16:01:32.148804 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.148787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.151742 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.151713    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-89ce7e-serving-cert\""
Jul 15 16:01:32.166576 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.166553    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"]
Jul 15 16:01:32.242544 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.242517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ae2130-f36e-487c-9978-00de65671278-maas-api-tls\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.242713 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.242559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mxtk\" (UniqueName: \"kubernetes.io/projected/88ae2130-f36e-487c-9978-00de65671278-kube-api-access-6mxtk\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.242713 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.242629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ae2130-f36e-487c-9978-00de65671278-openshift-service-ca\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.294005 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.293975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:01:32.343666 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.343629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ae2130-f36e-487c-9978-00de65671278-openshift-service-ca\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.343850 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.343793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ae2130-f36e-487c-9978-00de65671278-maas-api-tls\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.343850 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.343844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mxtk\" (UniqueName: \"kubernetes.io/projected/88ae2130-f36e-487c-9978-00de65671278-kube-api-access-6mxtk\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.344315 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.344288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ae2130-f36e-487c-9978-00de65671278-openshift-service-ca\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.346266 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.346248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ae2130-f36e-487c-9978-00de65671278-maas-api-tls\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.365568 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.365546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mxtk\" (UniqueName: \"kubernetes.io/projected/88ae2130-f36e-487c-9978-00de65671278-kube-api-access-6mxtk\") pod \"maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.459470 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.459401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:32.606094 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:32.606063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"]
Jul 15 16:01:32.607602 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:01:32.607569    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ae2130_f36e_487c_9978_00de65671278.slice/crio-0b7fb9535005252edc7252d3ebb241f9117ecca2a0c5322b710380c9c6bc0e8d WatchSource:0}: Error finding container 0b7fb9535005252edc7252d3ebb241f9117ecca2a0c5322b710380c9c6bc0e8d: Status 404 returned error can't find the container with id 0b7fb9535005252edc7252d3ebb241f9117ecca2a0c5322b710380c9c6bc0e8d
Jul 15 16:01:33.053660 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:33.053628    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94" event={"ID":"88ae2130-f36e-487c-9978-00de65671278","Type":"ContainerStarted","Data":"0b7fb9535005252edc7252d3ebb241f9117ecca2a0c5322b710380c9c6bc0e8d"}
Jul 15 16:01:34.034855 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:34.034822    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:01:34.063589 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:34.063556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94" event={"ID":"88ae2130-f36e-487c-9978-00de65671278","Type":"ContainerStarted","Data":"4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064"}
Jul 15 16:01:34.063791 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:34.063617    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:34.087922 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:34.087864    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94" podStartSLOduration=1.624208307 podStartE2EDuration="2.087843082s" totalImagesPullingTime="463.634775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:01:34.085194017 +0000 UTC m=+2218.886653860" watchObservedRunningTime="2026-07-15 16:01:34.087843082 +0000 UTC m=+2218.889302899"
Jul 15 16:01:35.990823 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:35.990795    2580 scope.go:117] "RemoveContainer" containerID="4791e9f712ba7cbd33ba12be699a3f9bd4d52d9af392f26dd810842c3a19cae3"
Jul 15 16:01:40.074995 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:40.074964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:01:43.275799 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:43.275764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:01:47.670543 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.670511    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"]
Jul 15 16:01:47.673980 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.673964    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.676785 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.676743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a920e4\"/\"default-dockercfg-6jzrt\""
Jul 15 16:01:47.677865 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.677844    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a920e4\"/\"test-model-a920e4-kserve-self-signed-certs\""
Jul 15 16:01:47.677968 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.677844    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a920e4\"/\"kube-root-ca.crt\""
Jul 15 16:01:47.677968 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.677884    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a920e4\"/\"openshift-service-ca.crt\""
Jul 15 16:01:47.683004 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.682985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"]
Jul 15 16:01:47.792118 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.792084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbhvl\" (UniqueName: \"kubernetes.io/projected/07db962b-cd9f-4742-8d3a-bb94a7f875c0-kube-api-access-rbhvl\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.792118 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.792122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-model-cache\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.792322 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.792201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tls-certs\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.792322 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.792267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-home\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.792322 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.792296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-dshm\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.792435 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.792324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tmp-dir\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.893801 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.893771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbhvl\" (UniqueName: \"kubernetes.io/projected/07db962b-cd9f-4742-8d3a-bb94a7f875c0-kube-api-access-rbhvl\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.893801 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.893805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-model-cache\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.894026 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.893853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tls-certs\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.894026 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.893895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-home\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.894026 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.893919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-dshm\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.894026 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.893947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tmp-dir\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.894326 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.894305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-home\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.894393 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.894358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-model-cache\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.894455 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.894433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tmp-dir\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.896161 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.896138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-dshm\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.896396 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.896379    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tls-certs\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.902908 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.902885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbhvl\" (UniqueName: \"kubernetes.io/projected/07db962b-cd9f-4742-8d3a-bb94a7f875c0-kube-api-access-rbhvl\") pod \"test-model-a920e4-kserve-75f4b74fbc-8lvqd\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") " pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:47.983722 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:47.983659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:48.121849 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:48.121822    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"]
Jul 15 16:01:48.122875 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:01:48.122850    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07db962b_cd9f_4742_8d3a_bb94a7f875c0.slice/crio-ad18f0949537b758892d0f854fe2089facd853683b72565f161743a672946a28 WatchSource:0}: Error finding container ad18f0949537b758892d0f854fe2089facd853683b72565f161743a672946a28: Status 404 returned error can't find the container with id ad18f0949537b758892d0f854fe2089facd853683b72565f161743a672946a28
Jul 15 16:01:48.156642 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:48.156612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd" event={"ID":"07db962b-cd9f-4742-8d3a-bb94a7f875c0","Type":"ContainerStarted","Data":"ad18f0949537b758892d0f854fe2089facd853683b72565f161743a672946a28"}
Jul 15 16:01:49.164815 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:49.164779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd" event={"ID":"07db962b-cd9f-4742-8d3a-bb94a7f875c0","Type":"ContainerStarted","Data":"12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929"}
Jul 15 16:01:49.185544 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:49.185490    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd" podStartSLOduration=2.185474577 podStartE2EDuration="2.185474577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:01:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:01:49.183402024 +0000 UTC m=+2233.984861839" watchObservedRunningTime="2026-07-15 16:01:49.185474577 +0000 UTC m=+2233.986934392"
Jul 15 16:01:51.864666 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:51.864633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:01:57.984074 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:57.983966    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:57.984074 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:57.984042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:57.996619 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:57.996595    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:01:58.230340 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:01:58.230312    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:02:03.664953 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:03.664917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:02:10.271478 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.271439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"]
Jul 15 16:02:10.277195 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.277172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.281921 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.281640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-89ce7e\"/\"test-model-89ce7e-kserve-self-signed-certs\""
Jul 15 16:02:10.281921 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.281687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-89ce7e\"/\"kube-root-ca.crt\""
Jul 15 16:02:10.281921 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.281736    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-89ce7e\"/\"default-dockercfg-8d6nd\""
Jul 15 16:02:10.281921 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.281844    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-89ce7e\"/\"openshift-service-ca.crt\""
Jul 15 16:02:10.285309 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.285286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"]
Jul 15 16:02:10.317169 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.317137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-home\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.317307 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.317255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-model-cache\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.317307 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.317286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-dshm\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.317389 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.317353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtmkg\" (UniqueName: \"kubernetes.io/projected/43503394-9a13-4ce2-94bd-a1f810e92cd1-kube-api-access-gtmkg\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.317389 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.317378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-tmp-dir\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.317464 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.317429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43503394-9a13-4ce2-94bd-a1f810e92cd1-tls-certs\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.418644 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.418609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-home\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.418815 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.418704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-model-cache\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.418815 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.418734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-dshm\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.418815 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.418793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtmkg\" (UniqueName: \"kubernetes.io/projected/43503394-9a13-4ce2-94bd-a1f810e92cd1-kube-api-access-gtmkg\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.418958 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.418815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-tmp-dir\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.418958 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.418870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43503394-9a13-4ce2-94bd-a1f810e92cd1-tls-certs\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.419084 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.419056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-home\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.419145 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.419079    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-model-cache\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.419237 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.419220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-tmp-dir\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.421022 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.421002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-dshm\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.421161 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.421143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43503394-9a13-4ce2-94bd-a1f810e92cd1-tls-certs\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.431070 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.431047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtmkg\" (UniqueName: \"kubernetes.io/projected/43503394-9a13-4ce2-94bd-a1f810e92cd1-kube-api-access-gtmkg\") pod \"test-model-89ce7e-kserve-577bf969c5-qxjbz\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") " pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.590486 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.590453    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:10.730258 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:10.730227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"]
Jul 15 16:02:10.731136 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:02:10.731115    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43503394_9a13_4ce2_94bd_a1f810e92cd1.slice/crio-380d7b5c7652437d7673ca0a0749af04903fd3fe3917605d163bd2062731c898 WatchSource:0}: Error finding container 380d7b5c7652437d7673ca0a0749af04903fd3fe3917605d163bd2062731c898: Status 404 returned error can't find the container with id 380d7b5c7652437d7673ca0a0749af04903fd3fe3917605d163bd2062731c898
Jul 15 16:02:11.298589 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:11.298552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz" event={"ID":"43503394-9a13-4ce2-94bd-a1f810e92cd1","Type":"ContainerStarted","Data":"dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b"}
Jul 15 16:02:11.299000 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:11.298598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz" event={"ID":"43503394-9a13-4ce2-94bd-a1f810e92cd1","Type":"ContainerStarted","Data":"380d7b5c7652437d7673ca0a0749af04903fd3fe3917605d163bd2062731c898"}
Jul 15 16:02:11.318815 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:11.318770    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz" podStartSLOduration=1.318739317 podStartE2EDuration="1.318739317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:02:11.317310842 +0000 UTC m=+2256.118770657" watchObservedRunningTime="2026-07-15 16:02:11.318739317 +0000 UTC m=+2256.120199131"
Jul 15 16:02:14.974833 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:14.974805    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:02:20.591244 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:20.591209    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:20.591788 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:20.591262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:20.603783 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:20.603740    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:21.370455 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:21.370429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:29.071188 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:29.071155    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:02:29.884413 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:29.884377    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"]
Jul 15 16:02:30.956039 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:30.956007    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"]
Jul 15 16:02:41.075360 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:41.075324    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:02:41.550897 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:41.550811    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"]
Jul 15 16:02:41.551079 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:41.551056    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj" podUID="211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" containerName="maas-api" containerID="cri-o://5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6" gracePeriod=30
Jul 15 16:02:41.996638 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:41.996615    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:02:42.037929 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.037899    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-maas-api-tls\") pod \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") "
Jul 15 16:02:42.038086 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.037950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-openshift-service-ca\") pod \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") "
Jul 15 16:02:42.038086 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.037979    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfp72\" (UniqueName: \"kubernetes.io/projected/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-kube-api-access-mfp72\") pod \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\" (UID: \"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d\") "
Jul 15 16:02:42.038335 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.038298    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" (UID: "211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:02:42.040111 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.040075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" (UID: "211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:02:42.040207 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.040142    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-kube-api-access-mfp72" (OuterVolumeSpecName: "kube-api-access-mfp72") pod "211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" (UID: "211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d"). InnerVolumeSpecName "kube-api-access-mfp72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:02:42.139374 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.139295    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:42.139374 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.139324    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfp72\" (UniqueName: \"kubernetes.io/projected/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-kube-api-access-mfp72\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:42.139374 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.139338    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:42.493423 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.493338    2580 generic.go:356] "Generic (PLEG): container finished" podID="211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" containerID="5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6" exitCode=0
Jul 15 16:02:42.493423 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.493403    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"
Jul 15 16:02:42.493622 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.493425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj" event={"ID":"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d","Type":"ContainerDied","Data":"5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6"}
Jul 15 16:02:42.493622 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.493473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj" event={"ID":"211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d","Type":"ContainerDied","Data":"cd9bd4e4f47aff4700afdf09ec1fa26338e9dbb994664de6320cc97cd2d31709"}
Jul 15 16:02:42.493622 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.493495    2580 scope.go:117] "RemoveContainer" containerID="5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6"
Jul 15 16:02:42.504232 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.504216    2580 scope.go:117] "RemoveContainer" containerID="5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6"
Jul 15 16:02:42.504485 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:02:42.504463    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6\": container with ID starting with 5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6 not found: ID does not exist" containerID="5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6"
Jul 15 16:02:42.504565 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.504498    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6"} err="failed to get container status \"5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6\": rpc error: code = NotFound desc = could not find container \"5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6\": container with ID starting with 5e243c02ee39df49f44446e7d10f4025fadcf9b2b97611ba5046d5b16f675ba6 not found: ID does not exist"
Jul 15 16:02:42.516909 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.516884    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"]
Jul 15 16:02:42.519516 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:42.519495    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a920e4-7d559868b6-2mtbj"]
Jul 15 16:02:43.768436 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:43.768400    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" path="/var/lib/kubelet/pods/211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d/volumes"
Jul 15 16:02:44.905065 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:44.905026    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd" podUID="07db962b-cd9f-4742-8d3a-bb94a7f875c0" containerName="main" containerID="cri-o://12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929" gracePeriod=45
Jul 15 16:02:45.162697 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.162643    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:02:45.271635 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271602    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-model-cache\") pod \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") "
Jul 15 16:02:45.271635 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271637    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-home\") pod \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") "
Jul 15 16:02:45.271842 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271665    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-dshm\") pod \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") "
Jul 15 16:02:45.271842 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271722    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbhvl\" (UniqueName: \"kubernetes.io/projected/07db962b-cd9f-4742-8d3a-bb94a7f875c0-kube-api-access-rbhvl\") pod \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") "
Jul 15 16:02:45.271842 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tls-certs\") pod \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") "
Jul 15 16:02:45.271999 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271855    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tmp-dir\") pod \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\" (UID: \"07db962b-cd9f-4742-8d3a-bb94a7f875c0\") "
Jul 15 16:02:45.271999 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271930    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-home" (OuterVolumeSpecName: "home") pod "07db962b-cd9f-4742-8d3a-bb94a7f875c0" (UID: "07db962b-cd9f-4742-8d3a-bb94a7f875c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:45.271999 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.271938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "07db962b-cd9f-4742-8d3a-bb94a7f875c0" (UID: "07db962b-cd9f-4742-8d3a-bb94a7f875c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:45.272182 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.272156    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07db962b-cd9f-4742-8d3a-bb94a7f875c0" (UID: "07db962b-cd9f-4742-8d3a-bb94a7f875c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:45.272232 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.272222    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:45.272267 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.272240    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:45.274017 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.273996    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07db962b-cd9f-4742-8d3a-bb94a7f875c0" (UID: "07db962b-cd9f-4742-8d3a-bb94a7f875c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:02:45.274256 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.274233    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-dshm" (OuterVolumeSpecName: "dshm") pod "07db962b-cd9f-4742-8d3a-bb94a7f875c0" (UID: "07db962b-cd9f-4742-8d3a-bb94a7f875c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:45.274351 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.274238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07db962b-cd9f-4742-8d3a-bb94a7f875c0-kube-api-access-rbhvl" (OuterVolumeSpecName: "kube-api-access-rbhvl") pod "07db962b-cd9f-4742-8d3a-bb94a7f875c0" (UID: "07db962b-cd9f-4742-8d3a-bb94a7f875c0"). InnerVolumeSpecName "kube-api-access-rbhvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:02:45.373343 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.373313    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:45.373343 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.373339    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbhvl\" (UniqueName: \"kubernetes.io/projected/07db962b-cd9f-4742-8d3a-bb94a7f875c0-kube-api-access-rbhvl\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:45.373512 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.373349    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:45.373512 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.373358    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07db962b-cd9f-4742-8d3a-bb94a7f875c0-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:45.514654 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.514575    2580 generic.go:356] "Generic (PLEG): container finished" podID="07db962b-cd9f-4742-8d3a-bb94a7f875c0" containerID="12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929" exitCode=0
Jul 15 16:02:45.514654 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.514615    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"
Jul 15 16:02:45.514871 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.514663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd" event={"ID":"07db962b-cd9f-4742-8d3a-bb94a7f875c0","Type":"ContainerDied","Data":"12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929"}
Jul 15 16:02:45.514871 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.514703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd" event={"ID":"07db962b-cd9f-4742-8d3a-bb94a7f875c0","Type":"ContainerDied","Data":"ad18f0949537b758892d0f854fe2089facd853683b72565f161743a672946a28"}
Jul 15 16:02:45.514871 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.514720    2580 scope.go:117] "RemoveContainer" containerID="12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929"
Jul 15 16:02:45.524722 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.524699    2580 scope.go:117] "RemoveContainer" containerID="12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929"
Jul 15 16:02:45.525017 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:02:45.524996    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929\": container with ID starting with 12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929 not found: ID does not exist" containerID="12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929"
Jul 15 16:02:45.525072 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.525027    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929"} err="failed to get container status \"12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929\": rpc error: code = NotFound desc = could not find container \"12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929\": container with ID starting with 12466f07eb22af67e9a54f03b923d86bcba6ce3750211dc90872527536a3a929 not found: ID does not exist"
Jul 15 16:02:45.539077 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.539055    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"]
Jul 15 16:02:45.542819 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.542797    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a920e4/test-model-a920e4-kserve-75f4b74fbc-8lvqd"]
Jul 15 16:02:45.768784 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.768684    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07db962b-cd9f-4742-8d3a-bb94a7f875c0" path="/var/lib/kubelet/pods/07db962b-cd9f-4742-8d3a-bb94a7f875c0/volumes"
Jul 15 16:02:45.976383 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:45.976350    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz" podUID="43503394-9a13-4ce2-94bd-a1f810e92cd1" containerName="main" containerID="cri-o://dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b" gracePeriod=45
Jul 15 16:02:46.233924 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.233901    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:46.382657 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.382574    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtmkg\" (UniqueName: \"kubernetes.io/projected/43503394-9a13-4ce2-94bd-a1f810e92cd1-kube-api-access-gtmkg\") pod \"43503394-9a13-4ce2-94bd-a1f810e92cd1\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") "
Jul 15 16:02:46.382657 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.382613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43503394-9a13-4ce2-94bd-a1f810e92cd1-tls-certs\") pod \"43503394-9a13-4ce2-94bd-a1f810e92cd1\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") "
Jul 15 16:02:46.382657 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.382657    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-home\") pod \"43503394-9a13-4ce2-94bd-a1f810e92cd1\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") "
Jul 15 16:02:46.382955 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.382707    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-dshm\") pod \"43503394-9a13-4ce2-94bd-a1f810e92cd1\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") "
Jul 15 16:02:46.382955 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.382773    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-tmp-dir\") pod \"43503394-9a13-4ce2-94bd-a1f810e92cd1\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") "
Jul 15 16:02:46.382955 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.382799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-model-cache\") pod \"43503394-9a13-4ce2-94bd-a1f810e92cd1\" (UID: \"43503394-9a13-4ce2-94bd-a1f810e92cd1\") "
Jul 15 16:02:46.383117 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.382969    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-home" (OuterVolumeSpecName: "home") pod "43503394-9a13-4ce2-94bd-a1f810e92cd1" (UID: "43503394-9a13-4ce2-94bd-a1f810e92cd1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:46.383160 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.383114    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43503394-9a13-4ce2-94bd-a1f810e92cd1" (UID: "43503394-9a13-4ce2-94bd-a1f810e92cd1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:46.383160 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.383147    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:46.383228 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.383156    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-model-cache" (OuterVolumeSpecName: "model-cache") pod "43503394-9a13-4ce2-94bd-a1f810e92cd1" (UID: "43503394-9a13-4ce2-94bd-a1f810e92cd1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:46.384943 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.384921    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43503394-9a13-4ce2-94bd-a1f810e92cd1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43503394-9a13-4ce2-94bd-a1f810e92cd1" (UID: "43503394-9a13-4ce2-94bd-a1f810e92cd1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:02:46.385043 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.384936    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43503394-9a13-4ce2-94bd-a1f810e92cd1-kube-api-access-gtmkg" (OuterVolumeSpecName: "kube-api-access-gtmkg") pod "43503394-9a13-4ce2-94bd-a1f810e92cd1" (UID: "43503394-9a13-4ce2-94bd-a1f810e92cd1"). InnerVolumeSpecName "kube-api-access-gtmkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:02:46.385043 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.384968    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-dshm" (OuterVolumeSpecName: "dshm") pod "43503394-9a13-4ce2-94bd-a1f810e92cd1" (UID: "43503394-9a13-4ce2-94bd-a1f810e92cd1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:02:46.483940 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.483902    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:46.483940 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.483932    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:46.483940 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.483945    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtmkg\" (UniqueName: \"kubernetes.io/projected/43503394-9a13-4ce2-94bd-a1f810e92cd1-kube-api-access-gtmkg\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:46.484180 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.483956    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43503394-9a13-4ce2-94bd-a1f810e92cd1-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:46.484180 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.483969    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43503394-9a13-4ce2-94bd-a1f810e92cd1-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:02:46.522157 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.522126    2580 generic.go:356] "Generic (PLEG): container finished" podID="43503394-9a13-4ce2-94bd-a1f810e92cd1" containerID="dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b" exitCode=0
Jul 15 16:02:46.522306 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.522169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"
Jul 15 16:02:46.522306 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.522212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz" event={"ID":"43503394-9a13-4ce2-94bd-a1f810e92cd1","Type":"ContainerDied","Data":"dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b"}
Jul 15 16:02:46.522306 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.522255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz" event={"ID":"43503394-9a13-4ce2-94bd-a1f810e92cd1","Type":"ContainerDied","Data":"380d7b5c7652437d7673ca0a0749af04903fd3fe3917605d163bd2062731c898"}
Jul 15 16:02:46.522306 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.522277    2580 scope.go:117] "RemoveContainer" containerID="dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b"
Jul 15 16:02:46.532064 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.532046    2580 scope.go:117] "RemoveContainer" containerID="dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b"
Jul 15 16:02:46.532314 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:02:46.532295    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b\": container with ID starting with dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b not found: ID does not exist" containerID="dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b"
Jul 15 16:02:46.532380 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.532322    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b"} err="failed to get container status \"dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b\": rpc error: code = NotFound desc = could not find container \"dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b\": container with ID starting with dfe5645b2cb21656919688e1feb7e9126c2f4f7d6159487de98bfa9d75234c5b not found: ID does not exist"
Jul 15 16:02:46.545105 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.545080    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"]
Jul 15 16:02:46.550561 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:46.550538    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-89ce7e/test-model-89ce7e-kserve-577bf969c5-qxjbz"]
Jul 15 16:02:47.767790 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:47.767745    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43503394-9a13-4ce2-94bd-a1f810e92cd1" path="/var/lib/kubelet/pods/43503394-9a13-4ce2-94bd-a1f810e92cd1/volumes"
Jul 15 16:02:50.971767 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:50.971721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:02:57.246148 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246116    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"]
Jul 15 16:02:57.246536 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246523    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43503394-9a13-4ce2-94bd-a1f810e92cd1" containerName="main"
Jul 15 16:02:57.246536 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246537    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43503394-9a13-4ce2-94bd-a1f810e92cd1" containerName="main"
Jul 15 16:02:57.246610 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246550    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07db962b-cd9f-4742-8d3a-bb94a7f875c0" containerName="main"
Jul 15 16:02:57.246610 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246555    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07db962b-cd9f-4742-8d3a-bb94a7f875c0" containerName="main"
Jul 15 16:02:57.246610 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246570    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" containerName="maas-api"
Jul 15 16:02:57.246610 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246576    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" containerName="maas-api"
Jul 15 16:02:57.246734 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246641    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07db962b-cd9f-4742-8d3a-bb94a7f875c0" containerName="main"
Jul 15 16:02:57.246734 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246650    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="43503394-9a13-4ce2-94bd-a1f810e92cd1" containerName="main"
Jul 15 16:02:57.246734 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.246658    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="211dc51e-a8ec-4fbd-a8b7-a6ceb0a2d69d" containerName="maas-api"
Jul 15 16:02:57.251255 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.251235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"
Jul 15 16:02:57.253885 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.253867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 16:02:57.259280 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.259260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"]
Jul 15 16:02:57.379603 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.379572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c50ad3-f7a4-4fd6-be3b-7341481ddc79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p\" (UID: \"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"
Jul 15 16:02:57.480305 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.480274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c50ad3-f7a4-4fd6-be3b-7341481ddc79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p\" (UID: \"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"
Jul 15 16:02:57.480904 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.480883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c50ad3-f7a4-4fd6-be3b-7341481ddc79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p\" (UID: \"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"
Jul 15 16:02:57.563149 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.563074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"
Jul 15 16:02:57.726401 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:57.726376    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"]
Jul 15 16:02:57.729181 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:02:57.729156    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c50ad3_f7a4_4fd6_be3b_7341481ddc79.slice/crio-37fdc5d575a712a06c7798215a2865982460b0bf18829280a3c9113e5f904449 WatchSource:0}: Error finding container 37fdc5d575a712a06c7798215a2865982460b0bf18829280a3c9113e5f904449: Status 404 returned error can't find the container with id 37fdc5d575a712a06c7798215a2865982460b0bf18829280a3c9113e5f904449
Jul 15 16:02:58.596894 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:58.596862    2580 generic.go:356] "Generic (PLEG): container finished" podID="a5c50ad3-f7a4-4fd6-be3b-7341481ddc79" containerID="0ccea2a658b4c79dd299176681bfbbc203542e4310f3e4b3fbc5d24f9cf7cb74" exitCode=0
Jul 15 16:02:58.597285 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:58.596930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p" event={"ID":"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79","Type":"ContainerDied","Data":"0ccea2a658b4c79dd299176681bfbbc203542e4310f3e4b3fbc5d24f9cf7cb74"}
Jul 15 16:02:58.597285 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:58.596968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p" event={"ID":"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79","Type":"ContainerStarted","Data":"37fdc5d575a712a06c7798215a2865982460b0bf18829280a3c9113e5f904449"}
Jul 15 16:02:59.733569 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:59.733546    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"
Jul 15 16:02:59.903411 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:59.903332    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c50ad3-f7a4-4fd6-be3b-7341481ddc79-maas-api-service-ca\") pod \"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79\" (UID: \"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79\") "
Jul 15 16:02:59.903681 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:02:59.903659    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c50ad3-f7a4-4fd6-be3b-7341481ddc79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5c50ad3-f7a4-4fd6-be3b-7341481ddc79" (UID: "a5c50ad3-f7a4-4fd6-be3b-7341481ddc79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:03:00.004333 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:00.004303    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c50ad3-f7a4-4fd6-be3b-7341481ddc79-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:03:00.610782 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:00.610736    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p"
Jul 15 16:03:00.610957 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:00.610781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p" event={"ID":"a5c50ad3-f7a4-4fd6-be3b-7341481ddc79","Type":"ContainerDied","Data":"37fdc5d575a712a06c7798215a2865982460b0bf18829280a3c9113e5f904449"}
Jul 15 16:03:00.610957 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:00.610836    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37fdc5d575a712a06c7798215a2865982460b0bf18829280a3c9113e5f904449"
Jul 15 16:03:07.711247 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.711214    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"]
Jul 15 16:03:07.711773 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.711439    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94" podUID="88ae2130-f36e-487c-9978-00de65671278" containerName="maas-api" containerID="cri-o://4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064" gracePeriod=30
Jul 15 16:03:07.960785 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.960760    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:03:07.977434 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.977382    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ae2130-f36e-487c-9978-00de65671278-maas-api-tls\") pod \"88ae2130-f36e-487c-9978-00de65671278\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") "
Jul 15 16:03:07.977540 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.977456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ae2130-f36e-487c-9978-00de65671278-openshift-service-ca\") pod \"88ae2130-f36e-487c-9978-00de65671278\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") "
Jul 15 16:03:07.977540 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.977502    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mxtk\" (UniqueName: \"kubernetes.io/projected/88ae2130-f36e-487c-9978-00de65671278-kube-api-access-6mxtk\") pod \"88ae2130-f36e-487c-9978-00de65671278\" (UID: \"88ae2130-f36e-487c-9978-00de65671278\") "
Jul 15 16:03:07.977806 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.977784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88ae2130-f36e-487c-9978-00de65671278-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "88ae2130-f36e-487c-9978-00de65671278" (UID: "88ae2130-f36e-487c-9978-00de65671278"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:03:07.979469 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.979443    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ae2130-f36e-487c-9978-00de65671278-kube-api-access-6mxtk" (OuterVolumeSpecName: "kube-api-access-6mxtk") pod "88ae2130-f36e-487c-9978-00de65671278" (UID: "88ae2130-f36e-487c-9978-00de65671278"). InnerVolumeSpecName "kube-api-access-6mxtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:03:07.979469 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:07.979463    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88ae2130-f36e-487c-9978-00de65671278-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "88ae2130-f36e-487c-9978-00de65671278" (UID: "88ae2130-f36e-487c-9978-00de65671278"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:03:08.078470 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.078437    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88ae2130-f36e-487c-9978-00de65671278-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:03:08.078470 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.078468    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mxtk\" (UniqueName: \"kubernetes.io/projected/88ae2130-f36e-487c-9978-00de65671278-kube-api-access-6mxtk\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:03:08.078639 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.078485    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88ae2130-f36e-487c-9978-00de65671278-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:03:08.658716 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.658683    2580 generic.go:356] "Generic (PLEG): container finished" podID="88ae2130-f36e-487c-9978-00de65671278" containerID="4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064" exitCode=0
Jul 15 16:03:08.658904 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.658727    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"
Jul 15 16:03:08.658904 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.658787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94" event={"ID":"88ae2130-f36e-487c-9978-00de65671278","Type":"ContainerDied","Data":"4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064"}
Jul 15 16:03:08.658904 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.658843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94" event={"ID":"88ae2130-f36e-487c-9978-00de65671278","Type":"ContainerDied","Data":"0b7fb9535005252edc7252d3ebb241f9117ecca2a0c5322b710380c9c6bc0e8d"}
Jul 15 16:03:08.658904 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.658872    2580 scope.go:117] "RemoveContainer" containerID="4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064"
Jul 15 16:03:08.668698 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.668677    2580 scope.go:117] "RemoveContainer" containerID="4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064"
Jul 15 16:03:08.668957 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:03:08.668937    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064\": container with ID starting with 4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064 not found: ID does not exist" containerID="4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064"
Jul 15 16:03:08.669005 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.668966    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064"} err="failed to get container status \"4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064\": rpc error: code = NotFound desc = could not find container \"4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064\": container with ID starting with 4009cacaf266f0acbb4d928744b8a9c926d46b0c60410fc387350d8994e34064 not found: ID does not exist"
Jul 15 16:03:08.681912 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.681885    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"]
Jul 15 16:03:08.685660 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:08.685638    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-89ce7e-56b49dbd79-xjr94"]
Jul 15 16:03:09.767589 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:09.767555    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88ae2130-f36e-487c-9978-00de65671278" path="/var/lib/kubelet/pods/88ae2130-f36e-487c-9978-00de65671278/volumes"
Jul 15 16:03:10.775807 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:10.775776    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:03:27.548615 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.548521    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"]
Jul 15 16:03:27.549165 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.549139    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ae2130-f36e-487c-9978-00de65671278" containerName="maas-api"
Jul 15 16:03:27.549165 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.549161    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ae2130-f36e-487c-9978-00de65671278" containerName="maas-api"
Jul 15 16:03:27.549260 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.549203    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c50ad3-f7a4-4fd6-be3b-7341481ddc79" containerName="revoke-keys"
Jul 15 16:03:27.549260 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.549212    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c50ad3-f7a4-4fd6-be3b-7341481ddc79" containerName="revoke-keys"
Jul 15 16:03:27.549336 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.549302    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="88ae2130-f36e-487c-9978-00de65671278" containerName="maas-api"
Jul 15 16:03:27.549336 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.549314    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c50ad3-f7a4-4fd6-be3b-7341481ddc79" containerName="revoke-keys"
Jul 15 16:03:27.552518 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.552500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.555408 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.555387    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 16:03:27.555855 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.555832    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:03:27.556088 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.556056    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-097eff-serving-cert\""
Jul 15 16:03:27.556528 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.556507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 16:03:27.565034 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.565012    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"]
Jul 15 16:03:27.657342 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.657295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfg2b\" (UniqueName: \"kubernetes.io/projected/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-kube-api-access-tfg2b\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.657342 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.657348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.657549 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.657417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-openshift-service-ca\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.758574 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.758526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-openshift-service-ca\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.758742 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.758620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfg2b\" (UniqueName: \"kubernetes.io/projected/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-kube-api-access-tfg2b\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.758742 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.758642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.758742 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:03:27.758736    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-serving-cert: secret "maas-api-e2e-api-a-097eff-serving-cert" not found
Jul 15 16:03:27.758893 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:03:27.758820    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls podName:413e5fe4-a2de-4d50-b23c-1c1b4334aad3 nodeName:}" failed. No retries permitted until 2026-07-15 16:03:28.258800636 +0000 UTC m=+2333.060260428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls") pod "maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt" (UID: "413e5fe4-a2de-4d50-b23c-1c1b4334aad3") : secret "maas-api-e2e-api-a-097eff-serving-cert" not found
Jul 15 16:03:27.759154 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.759135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-openshift-service-ca\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:27.773273 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:27.773240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfg2b\" (UniqueName: \"kubernetes.io/projected/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-kube-api-access-tfg2b\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:28.263498 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:28.263460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:28.266103 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:28.266080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls\") pod \"maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:28.464811 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:28.464776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:28.604066 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:28.604038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"]
Jul 15 16:03:28.605327 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:03:28.605303    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod413e5fe4_a2de_4d50_b23c_1c1b4334aad3.slice/crio-1fb87eb768159b9134f7288c84c9e46ad0aacbb9408925a623cd562e8b6f81c1 WatchSource:0}: Error finding container 1fb87eb768159b9134f7288c84c9e46ad0aacbb9408925a623cd562e8b6f81c1: Status 404 returned error can't find the container with id 1fb87eb768159b9134f7288c84c9e46ad0aacbb9408925a623cd562e8b6f81c1
Jul 15 16:03:28.607037 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:28.607013    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 16:03:28.782978 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:28.782895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt" event={"ID":"413e5fe4-a2de-4d50-b23c-1c1b4334aad3","Type":"ContainerStarted","Data":"1fb87eb768159b9134f7288c84c9e46ad0aacbb9408925a623cd562e8b6f81c1"}
Jul 15 16:03:30.795195 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:30.795106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt" event={"ID":"413e5fe4-a2de-4d50-b23c-1c1b4334aad3","Type":"ContainerStarted","Data":"5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1"}
Jul 15 16:03:30.795578 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:30.795215    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:30.818978 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:30.818933    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt" podStartSLOduration=1.9936536980000001 podStartE2EDuration="3.818917989s" totalImagesPullingTime="1.825264291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:03:30.81435291 +0000 UTC m=+2335.615812726" watchObservedRunningTime="2026-07-15 16:03:30.818917989 +0000 UTC m=+2335.620377802"
Jul 15 16:03:31.378738 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:31.378703    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:03:36.807076 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:36.807042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:03:37.962689 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:37.962657    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"]
Jul 15 16:03:37.968052 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:37.968034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:37.973952 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:37.973934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-552a11-serving-cert\""
Jul 15 16:03:37.993635 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:37.993603    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"]
Jul 15 16:03:38.057249 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.057202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e99306c-e70b-442e-8113-3ae22a47a4f9-openshift-service-ca\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.057249 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.057243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gzp\" (UniqueName: \"kubernetes.io/projected/9e99306c-e70b-442e-8113-3ae22a47a4f9-kube-api-access-v9gzp\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.057452 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.057335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.158190 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.158153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.158339 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.158225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e99306c-e70b-442e-8113-3ae22a47a4f9-openshift-service-ca\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.158339 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.158244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gzp\" (UniqueName: \"kubernetes.io/projected/9e99306c-e70b-442e-8113-3ae22a47a4f9-kube-api-access-v9gzp\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.158339 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:03:38.158298    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-serving-cert: secret "maas-api-e2e-api-b-552a11-serving-cert" not found
Jul 15 16:03:38.158485 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:03:38.158371    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls podName:9e99306c-e70b-442e-8113-3ae22a47a4f9 nodeName:}" failed. No retries permitted until 2026-07-15 16:03:38.658352708 +0000 UTC m=+2343.459812521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls") pod "maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69" (UID: "9e99306c-e70b-442e-8113-3ae22a47a4f9") : secret "maas-api-e2e-api-b-552a11-serving-cert" not found
Jul 15 16:03:38.158874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.158855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e99306c-e70b-442e-8113-3ae22a47a4f9-openshift-service-ca\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.173932 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.173903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gzp\" (UniqueName: \"kubernetes.io/projected/9e99306c-e70b-442e-8113-3ae22a47a4f9-kube-api-access-v9gzp\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.662833 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.662798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.665152 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.665126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls\") pod \"maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:38.878351 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:38.878324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:39.038299 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:39.038268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"]
Jul 15 16:03:39.038820 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:03:39.038796    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e99306c_e70b_442e_8113_3ae22a47a4f9.slice/crio-bae083e6734afbb122f8ac00bb367db933171533499b13e07d354b222f011328 WatchSource:0}: Error finding container bae083e6734afbb122f8ac00bb367db933171533499b13e07d354b222f011328: Status 404 returned error can't find the container with id bae083e6734afbb122f8ac00bb367db933171533499b13e07d354b222f011328
Jul 15 16:03:39.856328 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:39.856295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69" event={"ID":"9e99306c-e70b-442e-8113-3ae22a47a4f9","Type":"ContainerStarted","Data":"b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707"}
Jul 15 16:03:39.856490 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:39.856350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69" event={"ID":"9e99306c-e70b-442e-8113-3ae22a47a4f9","Type":"ContainerStarted","Data":"bae083e6734afbb122f8ac00bb367db933171533499b13e07d354b222f011328"}
Jul 15 16:03:39.856490 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:39.856383    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:39.879074 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:39.879022    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69" podStartSLOduration=2.544297786 podStartE2EDuration="2.879007742s" totalImagesPullingTime="334.709956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:03:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:03:39.875310803 +0000 UTC m=+2344.676770617" watchObservedRunningTime="2026-07-15 16:03:39.879007742 +0000 UTC m=+2344.680467558"
Jul 15 16:03:45.571203 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:45.571170    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:03:45.867443 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:45.867368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:03:50.297258 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.297224    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"]
Jul 15 16:03:50.300823 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.300805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"
Jul 15 16:03:50.303502 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.303481    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 16:03:50.309361 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.309328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"]
Jul 15 16:03:50.470317 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.470280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa0d377-0472-4814-b61d-066ac5c89f97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-097eff-j4rp8\" (UID: \"5aa0d377-0472-4814-b61d-066ac5c89f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"
Jul 15 16:03:50.571045 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.571016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa0d377-0472-4814-b61d-066ac5c89f97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-097eff-j4rp8\" (UID: \"5aa0d377-0472-4814-b61d-066ac5c89f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"
Jul 15 16:03:50.571677 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.571656    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa0d377-0472-4814-b61d-066ac5c89f97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-097eff-j4rp8\" (UID: \"5aa0d377-0472-4814-b61d-066ac5c89f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"
Jul 15 16:03:50.614944 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.614909    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"
Jul 15 16:03:50.747138 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.747106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"]
Jul 15 16:03:50.747949 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:03:50.747928    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa0d377_0472_4814_b61d_066ac5c89f97.slice/crio-f5fc911ce0b905a1dfd4e4678b886481247f6d5da200a917739158bf53ba0076 WatchSource:0}: Error finding container f5fc911ce0b905a1dfd4e4678b886481247f6d5da200a917739158bf53ba0076: Status 404 returned error can't find the container with id f5fc911ce0b905a1dfd4e4678b886481247f6d5da200a917739158bf53ba0076
Jul 15 16:03:50.924325 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:50.924293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8" event={"ID":"5aa0d377-0472-4814-b61d-066ac5c89f97","Type":"ContainerStarted","Data":"f5fc911ce0b905a1dfd4e4678b886481247f6d5da200a917739158bf53ba0076"}
Jul 15 16:03:51.931716 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:51.931683    2580 generic.go:356] "Generic (PLEG): container finished" podID="5aa0d377-0472-4814-b61d-066ac5c89f97" containerID="71cdc5807d0e6144f76d1a2d40522c4d51be28bc1010f90b8979e52c7d9b3868" exitCode=0
Jul 15 16:03:51.932122 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:51.931737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8" event={"ID":"5aa0d377-0472-4814-b61d-066ac5c89f97","Type":"ContainerDied","Data":"71cdc5807d0e6144f76d1a2d40522c4d51be28bc1010f90b8979e52c7d9b3868"}
Jul 15 16:03:53.069265 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:53.069243    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"
Jul 15 16:03:53.197384 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:53.197313    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa0d377-0472-4814-b61d-066ac5c89f97-maas-api-service-ca\") pod \"5aa0d377-0472-4814-b61d-066ac5c89f97\" (UID: \"5aa0d377-0472-4814-b61d-066ac5c89f97\") "
Jul 15 16:03:53.197643 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:53.197621    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5aa0d377-0472-4814-b61d-066ac5c89f97-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5aa0d377-0472-4814-b61d-066ac5c89f97" (UID: "5aa0d377-0472-4814-b61d-066ac5c89f97"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:03:53.298685 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:53.298658    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa0d377-0472-4814-b61d-066ac5c89f97-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:03:53.945237 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:53.945199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8" event={"ID":"5aa0d377-0472-4814-b61d-066ac5c89f97","Type":"ContainerDied","Data":"f5fc911ce0b905a1dfd4e4678b886481247f6d5da200a917739158bf53ba0076"}
Jul 15 16:03:53.945237 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:53.945228    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-097eff-j4rp8"
Jul 15 16:03:53.945432 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:03:53.945262    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5fc911ce0b905a1dfd4e4678b886481247f6d5da200a917739158bf53ba0076"
Jul 15 16:04:00.678713 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.678672    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"]
Jul 15 16:04:00.679317 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.678911    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt" podUID="413e5fe4-a2de-4d50-b23c-1c1b4334aad3" containerName="maas-api" containerID="cri-o://5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1" gracePeriod=30
Jul 15 16:04:00.935678 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.935619    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:04:00.966947 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.966919    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-openshift-service-ca\") pod \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") "
Jul 15 16:04:00.967084 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.967013    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls\") pod \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") "
Jul 15 16:04:00.967084 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.967042    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfg2b\" (UniqueName: \"kubernetes.io/projected/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-kube-api-access-tfg2b\") pod \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\" (UID: \"413e5fe4-a2de-4d50-b23c-1c1b4334aad3\") "
Jul 15 16:04:00.967348 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.967320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "413e5fe4-a2de-4d50-b23c-1c1b4334aad3" (UID: "413e5fe4-a2de-4d50-b23c-1c1b4334aad3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:04:00.969033 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.969011    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-kube-api-access-tfg2b" (OuterVolumeSpecName: "kube-api-access-tfg2b") pod "413e5fe4-a2de-4d50-b23c-1c1b4334aad3" (UID: "413e5fe4-a2de-4d50-b23c-1c1b4334aad3"). InnerVolumeSpecName "kube-api-access-tfg2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:04:00.969135 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.969044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "413e5fe4-a2de-4d50-b23c-1c1b4334aad3" (UID: "413e5fe4-a2de-4d50-b23c-1c1b4334aad3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:04:00.987332 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.987304    2580 generic.go:356] "Generic (PLEG): container finished" podID="413e5fe4-a2de-4d50-b23c-1c1b4334aad3" containerID="5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1" exitCode=0
Jul 15 16:04:00.987445 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.987354    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"
Jul 15 16:04:00.987445 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.987379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt" event={"ID":"413e5fe4-a2de-4d50-b23c-1c1b4334aad3","Type":"ContainerDied","Data":"5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1"}
Jul 15 16:04:00.987445 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.987437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt" event={"ID":"413e5fe4-a2de-4d50-b23c-1c1b4334aad3","Type":"ContainerDied","Data":"1fb87eb768159b9134f7288c84c9e46ad0aacbb9408925a623cd562e8b6f81c1"}
Jul 15 16:04:00.987568 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.987461    2580 scope.go:117] "RemoveContainer" containerID="5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1"
Jul 15 16:04:00.997907 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.997887    2580 scope.go:117] "RemoveContainer" containerID="5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1"
Jul 15 16:04:00.998160 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:04:00.998142    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1\": container with ID starting with 5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1 not found: ID does not exist" containerID="5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1"
Jul 15 16:04:00.998230 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:00.998168    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1"} err="failed to get container status \"5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1\": rpc error: code = NotFound desc = could not find container \"5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1\": container with ID starting with 5392981fd76d965e2984f86f492df0b9eedd67efedbf2b8238ca9100974f56a1 not found: ID does not exist"
Jul 15 16:04:01.014078 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:01.014052    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"]
Jul 15 16:04:01.018242 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:01.018214    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-097eff-5c69b6465c-q7mxt"]
Jul 15 16:04:01.067773 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:01.067732    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:04:01.067773 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:01.067775    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:04:01.067919 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:01.067786    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfg2b\" (UniqueName: \"kubernetes.io/projected/413e5fe4-a2de-4d50-b23c-1c1b4334aad3-kube-api-access-tfg2b\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:04:01.767673 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:01.767640    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="413e5fe4-a2de-4d50-b23c-1c1b4334aad3" path="/var/lib/kubelet/pods/413e5fe4-a2de-4d50-b23c-1c1b4334aad3/volumes"
Jul 15 16:04:04.372342 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:04.372306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:04:16.430856 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.430815    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"]
Jul 15 16:04:16.431259 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.431242    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413e5fe4-a2de-4d50-b23c-1c1b4334aad3" containerName="maas-api"
Jul 15 16:04:16.431299 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.431260    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="413e5fe4-a2de-4d50-b23c-1c1b4334aad3" containerName="maas-api"
Jul 15 16:04:16.431299 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.431269    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aa0d377-0472-4814-b61d-066ac5c89f97" containerName="revoke-keys"
Jul 15 16:04:16.431299 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.431274    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aa0d377-0472-4814-b61d-066ac5c89f97" containerName="revoke-keys"
Jul 15 16:04:16.431397 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.431343    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="413e5fe4-a2de-4d50-b23c-1c1b4334aad3" containerName="maas-api"
Jul 15 16:04:16.431397 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.431351    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aa0d377-0472-4814-b61d-066ac5c89f97" containerName="revoke-keys"
Jul 15 16:04:16.434347 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.434332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"
Jul 15 16:04:16.436743 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.436723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 16:04:16.441873 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.441847    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"]
Jul 15 16:04:16.512625 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.512585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554a0273-ac6f-40bd-9f1d-904b252fddc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-552a11-bx4vx\" (UID: \"554a0273-ac6f-40bd-9f1d-904b252fddc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"
Jul 15 16:04:16.613665 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.613620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554a0273-ac6f-40bd-9f1d-904b252fddc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-552a11-bx4vx\" (UID: \"554a0273-ac6f-40bd-9f1d-904b252fddc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"
Jul 15 16:04:16.614231 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.614209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554a0273-ac6f-40bd-9f1d-904b252fddc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-552a11-bx4vx\" (UID: \"554a0273-ac6f-40bd-9f1d-904b252fddc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"
Jul 15 16:04:16.746075 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.745989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"
Jul 15 16:04:16.880823 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:16.880792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"]
Jul 15 16:04:16.881794 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:04:16.881761    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554a0273_ac6f_40bd_9f1d_904b252fddc2.slice/crio-62393687cbf672dc78955c44437bdedacef7d2c1f713787c80ca2437039d9dba WatchSource:0}: Error finding container 62393687cbf672dc78955c44437bdedacef7d2c1f713787c80ca2437039d9dba: Status 404 returned error can't find the container with id 62393687cbf672dc78955c44437bdedacef7d2c1f713787c80ca2437039d9dba
Jul 15 16:04:17.084969 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:17.084928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx" event={"ID":"554a0273-ac6f-40bd-9f1d-904b252fddc2","Type":"ContainerStarted","Data":"62393687cbf672dc78955c44437bdedacef7d2c1f713787c80ca2437039d9dba"}
Jul 15 16:04:18.092468 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:18.092433    2580 generic.go:356] "Generic (PLEG): container finished" podID="554a0273-ac6f-40bd-9f1d-904b252fddc2" containerID="9303f8d74298468bb2f2830fed611ac29d1a9dea6c115c3b967e5d100e46b067" exitCode=0
Jul 15 16:04:18.092895 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:18.092517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx" event={"ID":"554a0273-ac6f-40bd-9f1d-904b252fddc2","Type":"ContainerDied","Data":"9303f8d74298468bb2f2830fed611ac29d1a9dea6c115c3b967e5d100e46b067"}
Jul 15 16:04:19.231746 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:19.231718    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"
Jul 15 16:04:19.339014 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:19.338977    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554a0273-ac6f-40bd-9f1d-904b252fddc2-maas-api-service-ca\") pod \"554a0273-ac6f-40bd-9f1d-904b252fddc2\" (UID: \"554a0273-ac6f-40bd-9f1d-904b252fddc2\") "
Jul 15 16:04:19.339343 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:19.339323    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554a0273-ac6f-40bd-9f1d-904b252fddc2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "554a0273-ac6f-40bd-9f1d-904b252fddc2" (UID: "554a0273-ac6f-40bd-9f1d-904b252fddc2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:04:19.440429 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:19.440364    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554a0273-ac6f-40bd-9f1d-904b252fddc2-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:04:20.104929 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:20.104902    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx"
Jul 15 16:04:20.105111 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:20.104944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-552a11-bx4vx" event={"ID":"554a0273-ac6f-40bd-9f1d-904b252fddc2","Type":"ContainerDied","Data":"62393687cbf672dc78955c44437bdedacef7d2c1f713787c80ca2437039d9dba"}
Jul 15 16:04:20.105111 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:20.104990    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62393687cbf672dc78955c44437bdedacef7d2c1f713787c80ca2437039d9dba"
Jul 15 16:04:26.873098 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:26.873064    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"]
Jul 15 16:04:26.873575 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:26.873302    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69" podUID="9e99306c-e70b-442e-8113-3ae22a47a4f9" containerName="maas-api" containerID="cri-o://b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707" gracePeriod=30
Jul 15 16:04:27.134131 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.134071    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:04:27.151086 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.151058    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e99306c-e70b-442e-8113-3ae22a47a4f9" containerID="b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707" exitCode=0
Jul 15 16:04:27.151243 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.151103    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"
Jul 15 16:04:27.151243 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.151107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69" event={"ID":"9e99306c-e70b-442e-8113-3ae22a47a4f9","Type":"ContainerDied","Data":"b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707"}
Jul 15 16:04:27.151581 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.151241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69" event={"ID":"9e99306c-e70b-442e-8113-3ae22a47a4f9","Type":"ContainerDied","Data":"bae083e6734afbb122f8ac00bb367db933171533499b13e07d354b222f011328"}
Jul 15 16:04:27.151581 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.151267    2580 scope.go:117] "RemoveContainer" containerID="b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707"
Jul 15 16:04:27.162449 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.162418    2580 scope.go:117] "RemoveContainer" containerID="b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707"
Jul 15 16:04:27.162726 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:04:27.162707    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707\": container with ID starting with b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707 not found: ID does not exist" containerID="b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707"
Jul 15 16:04:27.162807 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.162735    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707"} err="failed to get container status \"b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707\": rpc error: code = NotFound desc = could not find container \"b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707\": container with ID starting with b4d0a54ce09dd77d8b91d1f00182e3abbf4c37de3aa8413bd1b6e52c184b9707 not found: ID does not exist"
Jul 15 16:04:27.211819 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.211791    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls\") pod \"9e99306c-e70b-442e-8113-3ae22a47a4f9\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") "
Jul 15 16:04:27.211947 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.211878    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9gzp\" (UniqueName: \"kubernetes.io/projected/9e99306c-e70b-442e-8113-3ae22a47a4f9-kube-api-access-v9gzp\") pod \"9e99306c-e70b-442e-8113-3ae22a47a4f9\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") "
Jul 15 16:04:27.211947 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.211911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e99306c-e70b-442e-8113-3ae22a47a4f9-openshift-service-ca\") pod \"9e99306c-e70b-442e-8113-3ae22a47a4f9\" (UID: \"9e99306c-e70b-442e-8113-3ae22a47a4f9\") "
Jul 15 16:04:27.212328 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.212304    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e99306c-e70b-442e-8113-3ae22a47a4f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e99306c-e70b-442e-8113-3ae22a47a4f9" (UID: "9e99306c-e70b-442e-8113-3ae22a47a4f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:04:27.213932 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.213897    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e99306c-e70b-442e-8113-3ae22a47a4f9" (UID: "9e99306c-e70b-442e-8113-3ae22a47a4f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:04:27.214046 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.214018    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e99306c-e70b-442e-8113-3ae22a47a4f9-kube-api-access-v9gzp" (OuterVolumeSpecName: "kube-api-access-v9gzp") pod "9e99306c-e70b-442e-8113-3ae22a47a4f9" (UID: "9e99306c-e70b-442e-8113-3ae22a47a4f9"). InnerVolumeSpecName "kube-api-access-v9gzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:04:27.313260 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.313229    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e99306c-e70b-442e-8113-3ae22a47a4f9-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:04:27.313260 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.313257    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9gzp\" (UniqueName: \"kubernetes.io/projected/9e99306c-e70b-442e-8113-3ae22a47a4f9-kube-api-access-v9gzp\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:04:27.313403 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.313266    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e99306c-e70b-442e-8113-3ae22a47a4f9-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:04:27.474783 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.474745    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"]
Jul 15 16:04:27.483167 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.483141    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-552a11-5cdb65df5b-nxx69"]
Jul 15 16:04:27.767499 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:27.767426    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e99306c-e70b-442e-8113-3ae22a47a4f9" path="/var/lib/kubelet/pods/9e99306c-e70b-442e-8113-3ae22a47a4f9/volumes"
Jul 15 16:04:29.864364 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:29.864331    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:04:35.904361 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:35.904328    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 16:04:35.915085 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:35.915062    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 16:04:46.137007 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.136975    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"]
Jul 15 16:04:46.137574 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.137555    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554a0273-ac6f-40bd-9f1d-904b252fddc2" containerName="revoke-keys"
Jul 15 16:04:46.137660 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.137577    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="554a0273-ac6f-40bd-9f1d-904b252fddc2" containerName="revoke-keys"
Jul 15 16:04:46.137660 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.137607    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e99306c-e70b-442e-8113-3ae22a47a4f9" containerName="maas-api"
Jul 15 16:04:46.137660 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.137616    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e99306c-e70b-442e-8113-3ae22a47a4f9" containerName="maas-api"
Jul 15 16:04:46.137865 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.137711    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="554a0273-ac6f-40bd-9f1d-904b252fddc2" containerName="revoke-keys"
Jul 15 16:04:46.137865 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.137728    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e99306c-e70b-442e-8113-3ae22a47a4f9" containerName="maas-api"
Jul 15 16:04:46.140785 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.140741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.143416 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.143393    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0e8a6d-serving-cert\""
Jul 15 16:04:46.143510 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.143420    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:04:46.143510 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.143437    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 16:04:46.144348 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.144330    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tkkc9\""
Jul 15 16:04:46.160658 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.160638    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"]
Jul 15 16:04:46.189440 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.189420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mlgf\" (UniqueName: \"kubernetes.io/projected/8d58aec0-ad57-4f0e-a8f3-7af92192b434-kube-api-access-9mlgf\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.189535 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.189456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d58aec0-ad57-4f0e-a8f3-7af92192b434-maas-api-tls\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.189576 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.189563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d58aec0-ad57-4f0e-a8f3-7af92192b434-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.290366 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.290339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d58aec0-ad57-4f0e-a8f3-7af92192b434-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.290532 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.290406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mlgf\" (UniqueName: \"kubernetes.io/projected/8d58aec0-ad57-4f0e-a8f3-7af92192b434-kube-api-access-9mlgf\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.290532 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.290435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d58aec0-ad57-4f0e-a8f3-7af92192b434-maas-api-tls\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.291045 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.291022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d58aec0-ad57-4f0e-a8f3-7af92192b434-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.292857 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.292835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d58aec0-ad57-4f0e-a8f3-7af92192b434-maas-api-tls\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.304004 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.303981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mlgf\" (UniqueName: \"kubernetes.io/projected/8d58aec0-ad57-4f0e-a8f3-7af92192b434-kube-api-access-9mlgf\") pod \"maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.450876 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.450795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:46.603785 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:46.603760    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"]
Jul 15 16:04:46.604951 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:04:46.604919    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d58aec0_ad57_4f0e_a8f3_7af92192b434.slice/crio-77dfb2d2f3e6bc2993c67c3c5915bd41653b58c567e24b658ab339a31e3484e2 WatchSource:0}: Error finding container 77dfb2d2f3e6bc2993c67c3c5915bd41653b58c567e24b658ab339a31e3484e2: Status 404 returned error can't find the container with id 77dfb2d2f3e6bc2993c67c3c5915bd41653b58c567e24b658ab339a31e3484e2
Jul 15 16:04:47.279934 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:47.279850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng" event={"ID":"8d58aec0-ad57-4f0e-a8f3-7af92192b434","Type":"ContainerStarted","Data":"931b804debebb4aa92a879826a67b675d699b04cbd8862a91a08d13c91aba7bd"}
Jul 15 16:04:47.279934 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:47.279911    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:47.279934 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:47.279933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng" event={"ID":"8d58aec0-ad57-4f0e-a8f3-7af92192b434","Type":"ContainerStarted","Data":"77dfb2d2f3e6bc2993c67c3c5915bd41653b58c567e24b658ab339a31e3484e2"}
Jul 15 16:04:47.305453 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:47.305406    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng" podStartSLOduration=0.940988873 podStartE2EDuration="1.305391038s" totalImagesPullingTime="364.402165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:04:47.303059184 +0000 UTC m=+2412.104518999" watchObservedRunningTime="2026-07-15 16:04:47.305391038 +0000 UTC m=+2412.106850853"
Jul 15 16:04:51.768842 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:51.768813    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:04:53.292028 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:53.291992    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:04:54.777646 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.777614    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"]
Jul 15 16:04:54.781588 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.781565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.784743 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.784725    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bcb615-serving-cert\""
Jul 15 16:04:54.801566 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.801541    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"]
Jul 15 16:04:54.873628 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.873599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpss6\" (UniqueName: \"kubernetes.io/projected/33ebe502-ab66-4481-b216-7a405e57ba0c-kube-api-access-fpss6\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.873834 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.873634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.873834 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.873683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ebe502-ab66-4481-b216-7a405e57ba0c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.975040 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.975009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ebe502-ab66-4481-b216-7a405e57ba0c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.975220 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.975092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpss6\" (UniqueName: \"kubernetes.io/projected/33ebe502-ab66-4481-b216-7a405e57ba0c-kube-api-access-fpss6\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.975220 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.975111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.975353 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:04:54.975269    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-serving-cert: secret "maas-api-e2e-shared-b-bcb615-serving-cert" not found
Jul 15 16:04:54.975353 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:04:54.975317    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls podName:33ebe502-ab66-4481-b216-7a405e57ba0c nodeName:}" failed. No retries permitted until 2026-07-15 16:04:55.475302385 +0000 UTC m=+2420.276762176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls") pod "maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp" (UID: "33ebe502-ab66-4481-b216-7a405e57ba0c") : secret "maas-api-e2e-shared-b-bcb615-serving-cert" not found
Jul 15 16:04:54.975713 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.975690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ebe502-ab66-4481-b216-7a405e57ba0c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:54.985374 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:54.985351    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpss6\" (UniqueName: \"kubernetes.io/projected/33ebe502-ab66-4481-b216-7a405e57ba0c-kube-api-access-fpss6\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:55.479537 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:55.479500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:55.482179 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:55.482156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls\") pod \"maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:55.692489 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:55.692462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:55.830654 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:55.830508    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"]
Jul 15 16:04:55.832550 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:04:55.832521    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ebe502_ab66_4481_b216_7a405e57ba0c.slice/crio-b8f80b0cf91c842262e3f05afe0003a12146da6fbdd95292c4f6d71047c2323f WatchSource:0}: Error finding container b8f80b0cf91c842262e3f05afe0003a12146da6fbdd95292c4f6d71047c2323f: Status 404 returned error can't find the container with id b8f80b0cf91c842262e3f05afe0003a12146da6fbdd95292c4f6d71047c2323f
Jul 15 16:04:56.336335 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:56.336304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp" event={"ID":"33ebe502-ab66-4481-b216-7a405e57ba0c","Type":"ContainerStarted","Data":"47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a"}
Jul 15 16:04:56.336493 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:56.336354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp" event={"ID":"33ebe502-ab66-4481-b216-7a405e57ba0c","Type":"ContainerStarted","Data":"b8f80b0cf91c842262e3f05afe0003a12146da6fbdd95292c4f6d71047c2323f"}
Jul 15 16:04:56.336493 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:56.336387    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:04:56.353461 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:04:56.353416    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp" podStartSLOduration=1.961462901 podStartE2EDuration="2.35340098s" totalImagesPullingTime="391.938079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:04:56.352695753 +0000 UTC m=+2421.154155580" watchObservedRunningTime="2026-07-15 16:04:56.35340098 +0000 UTC m=+2421.154860794"
Jul 15 16:05:00.272811 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:00.272725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:05:02.347520 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:02.347490    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:05:16.363715 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.363685    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"]
Jul 15 16:05:16.367364 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.367334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.370539 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.370516    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"openshift-service-ca.crt\""
Jul 15 16:05:16.370683 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.370660    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"kube-root-ca.crt\""
Jul 15 16:05:16.371520 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.371501    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"default-dockercfg-lxlzj\""
Jul 15 16:05:16.371642 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.371571    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"auth-test-model-0e8a6d-kserve-self-signed-certs\""
Jul 15 16:05:16.374536 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.374509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tmp-dir\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.374730 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.374713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-model-cache\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.374906 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.374890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tls-certs\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.375110 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.375085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-home\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.375228 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.375137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-dshm\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.375228 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.375176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh27x\" (UniqueName: \"kubernetes.io/projected/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-kube-api-access-rh27x\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.379276 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.378802    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"]
Jul 15 16:05:16.476328 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.476297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh27x\" (UniqueName: \"kubernetes.io/projected/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-kube-api-access-rh27x\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.476507 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.476338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tmp-dir\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.476507 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.476363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-model-cache\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.476507 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.476389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tls-certs\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.476507 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.476433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-home\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.476507 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.476453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-dshm\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.477231 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.477099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tmp-dir\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.477231 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.477147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-model-cache\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.477429 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.477302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-home\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.485410 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.479609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-dshm\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.485410 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.480205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tls-certs\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.487063 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.487040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh27x\" (UniqueName: \"kubernetes.io/projected/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-kube-api-access-rh27x\") pod \"auth-test-model-0e8a6d-kserve-97555d475-zvtbn\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") " pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.678577 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.678491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:16.813335 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:16.813304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"]
Jul 15 16:05:16.814193 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:05:16.814162    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d0d29d7_13c0_4f76_9bbb_7e5b27456ab5.slice/crio-c38b63ffe98e6e11c9776f9a3ff076c3ec5cefe77118c23e99c8317edccffa8d WatchSource:0}: Error finding container c38b63ffe98e6e11c9776f9a3ff076c3ec5cefe77118c23e99c8317edccffa8d: Status 404 returned error can't find the container with id c38b63ffe98e6e11c9776f9a3ff076c3ec5cefe77118c23e99c8317edccffa8d
Jul 15 16:05:17.465357 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:17.465320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn" event={"ID":"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5","Type":"ContainerStarted","Data":"2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e"}
Jul 15 16:05:17.465357 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:17.465365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn" event={"ID":"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5","Type":"ContainerStarted","Data":"c38b63ffe98e6e11c9776f9a3ff076c3ec5cefe77118c23e99c8317edccffa8d"}
Jul 15 16:05:17.483611 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:17.483549    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn" podStartSLOduration=1.483528427 podStartE2EDuration="1.483528427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:05:17.48211273 +0000 UTC m=+2442.283572544" watchObservedRunningTime="2026-07-15 16:05:17.483528427 +0000 UTC m=+2442.284988244"
Jul 15 16:05:19.071772 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:19.071711    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:05:26.679459 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:26.679429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:26.679865 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:26.679477    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:26.691862 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:26.691839    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:27.537720 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:27.537692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:05:48.588261 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.588221    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"]
Jul 15 16:05:48.591954 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.591934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.595944 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.595925    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"kube-root-ca.crt\""
Jul 15 16:05:48.595944 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.595933    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"auth-test-model-bcb615-kserve-self-signed-certs\""
Jul 15 16:05:48.596135 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.595966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"default-dockercfg-2462r\""
Jul 15 16:05:48.596135 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.595925    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"openshift-service-ca.crt\""
Jul 15 16:05:48.610954 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.610928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"]
Jul 15 16:05:48.682103 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.682075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-home\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.682264 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.682112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-model-cache\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.682264 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.682142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzsns\" (UniqueName: \"kubernetes.io/projected/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-kube-api-access-vzsns\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.682264 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.682224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-dshm\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.682264 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.682264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tls-certs\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.682481 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.682352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tmp-dir\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783039 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tmp-dir\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783216 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-home\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783216 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-model-cache\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783216 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzsns\" (UniqueName: \"kubernetes.io/projected/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-kube-api-access-vzsns\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783216 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-dshm\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783450 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tls-certs\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783520 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tmp-dir\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783520 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-home\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.783666 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.783561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-model-cache\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.785524 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.785497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-dshm\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.785700 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.785684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tls-certs\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.794258 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.794227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzsns\" (UniqueName: \"kubernetes.io/projected/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-kube-api-access-vzsns\") pod \"auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") " pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:48.911096 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:48.911016    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:49.048996 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:49.048972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"]
Jul 15 16:05:49.051180 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:05:49.051149    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61ca1b35_3edc_40e9_8803_f0b3f5d1d70e.slice/crio-de27f51d97f01c23b3ace53a659d3cfc4ca4aa6881ff38aca2d2df49d45b58a5 WatchSource:0}: Error finding container de27f51d97f01c23b3ace53a659d3cfc4ca4aa6881ff38aca2d2df49d45b58a5: Status 404 returned error can't find the container with id de27f51d97f01c23b3ace53a659d3cfc4ca4aa6881ff38aca2d2df49d45b58a5
Jul 15 16:05:49.664512 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:49.664472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x" event={"ID":"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e","Type":"ContainerStarted","Data":"cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1"}
Jul 15 16:05:49.664512 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:49.664516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x" event={"ID":"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e","Type":"ContainerStarted","Data":"de27f51d97f01c23b3ace53a659d3cfc4ca4aa6881ff38aca2d2df49d45b58a5"}
Jul 15 16:05:49.681743 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:49.681694    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x" podStartSLOduration=1.6816753740000001 podStartE2EDuration="1.681675374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:05:49.680463182 +0000 UTC m=+2474.481923011" watchObservedRunningTime="2026-07-15 16:05:49.681675374 +0000 UTC m=+2474.483135190"
Jul 15 16:05:52.069188 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:52.069152    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:05:58.911820 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:58.911785    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:58.912232 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:58.911841    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:58.924625 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:58.924600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:05:59.746339 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:05:59.746309    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:06:05.772328 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:06:05.772297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:06:24.365049 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:06:24.365006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:06:41.777658 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:06:41.777588    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:06:58.905281 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:06:58.905246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:07:15.385526 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:07:15.385486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:07:34.174587 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:07:34.174536    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:07:45.140879 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:07:45.140838    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"]
Jul 15 16:07:45.998159 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:07:45.998115    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"]
Jul 15 16:07:50.975952 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:07:50.975911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:08:00.164252 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.164171    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn" podUID="2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" containerName="main" containerID="cri-o://2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e" gracePeriod=45
Jul 15 16:08:00.412834 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.412809    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:08:00.450451 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-model-cache\") pod \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") "
Jul 15 16:08:00.450451 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh27x\" (UniqueName: \"kubernetes.io/projected/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-kube-api-access-rh27x\") pod \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") "
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450466    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tls-certs\") pod \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") "
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450494    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-home\") pod \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") "
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450523    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tmp-dir\") pod \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") "
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-dshm\") pod \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\" (UID: \"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5\") "
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450665    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" (UID: "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-home" (OuterVolumeSpecName: "home") pod "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" (UID: "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450846    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" (UID: "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450948    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.450966    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:00.451010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.451008    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:00.453276 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.453255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-dshm" (OuterVolumeSpecName: "dshm") pod "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" (UID: "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:00.453357 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.453299    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" (UID: "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:08:00.453357 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.453314    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-kube-api-access-rh27x" (OuterVolumeSpecName: "kube-api-access-rh27x") pod "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" (UID: "2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5"). InnerVolumeSpecName "kube-api-access-rh27x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:08:00.488172 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.488147    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" containerID="2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e" exitCode=0
Jul 15 16:08:00.488285 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.488190    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"
Jul 15 16:08:00.488285 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.488201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn" event={"ID":"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5","Type":"ContainerDied","Data":"2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e"}
Jul 15 16:08:00.488285 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.488244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn" event={"ID":"2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5","Type":"ContainerDied","Data":"c38b63ffe98e6e11c9776f9a3ff076c3ec5cefe77118c23e99c8317edccffa8d"}
Jul 15 16:08:00.488285 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.488267    2580 scope.go:117] "RemoveContainer" containerID="2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e"
Jul 15 16:08:00.498110 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.498089    2580 scope.go:117] "RemoveContainer" containerID="2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e"
Jul 15 16:08:00.498377 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:08:00.498357    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e\": container with ID starting with 2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e not found: ID does not exist" containerID="2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e"
Jul 15 16:08:00.498441 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.498390    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e"} err="failed to get container status \"2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e\": rpc error: code = NotFound desc = could not find container \"2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e\": container with ID starting with 2c83b3c566bf48baa4b8e8188358966ca531d02eb2ece9c88b1231fcb2e2745e not found: ID does not exist"
Jul 15 16:08:00.510800 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.510775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"]
Jul 15 16:08:00.516320 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.516295    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/auth-test-model-0e8a6d-kserve-97555d475-zvtbn"]
Jul 15 16:08:00.552029 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.552007    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh27x\" (UniqueName: \"kubernetes.io/projected/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-kube-api-access-rh27x\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:00.552029 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.552027    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:00.552162 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:00.552035    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:01.018665 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.018629    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x" podUID="61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" containerName="main" containerID="cri-o://cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1" gracePeriod=45
Jul 15 16:08:01.277819 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.277746    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:08:01.360510 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360482    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tls-certs\") pod \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") "
Jul 15 16:08:01.360658 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360536    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzsns\" (UniqueName: \"kubernetes.io/projected/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-kube-api-access-vzsns\") pod \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") "
Jul 15 16:08:01.360658 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-dshm\") pod \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") "
Jul 15 16:08:01.360658 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360586    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-home\") pod \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") "
Jul 15 16:08:01.360658 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360604    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-model-cache\") pod \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") "
Jul 15 16:08:01.360658 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tmp-dir\") pod \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\" (UID: \"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e\") "
Jul 15 16:08:01.361000 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360930    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-home" (OuterVolumeSpecName: "home") pod "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" (UID: "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:01.361000 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360951    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" (UID: "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:01.361000 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.360915    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-model-cache" (OuterVolumeSpecName: "model-cache") pod "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" (UID: "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:01.362824 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.362790    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" (UID: "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:08:01.362824 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.362807    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-dshm" (OuterVolumeSpecName: "dshm") pod "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" (UID: "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:08:01.362975 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.362893    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-kube-api-access-vzsns" (OuterVolumeSpecName: "kube-api-access-vzsns") pod "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" (UID: "61ca1b35-3edc-40e9-8803-f0b3f5d1d70e"). InnerVolumeSpecName "kube-api-access-vzsns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:08:01.462195 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.462161    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:01.462195 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.462190    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzsns\" (UniqueName: \"kubernetes.io/projected/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-kube-api-access-vzsns\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:01.462195 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.462199    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:01.462195 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.462211    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:01.462465 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.462218    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:01.462465 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.462227    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:08:01.495436 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.495399    2580 generic.go:356] "Generic (PLEG): container finished" podID="61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" containerID="cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1" exitCode=0
Jul 15 16:08:01.495606 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.495442    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"
Jul 15 16:08:01.495606 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.495485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x" event={"ID":"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e","Type":"ContainerDied","Data":"cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1"}
Jul 15 16:08:01.495606 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.495526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x" event={"ID":"61ca1b35-3edc-40e9-8803-f0b3f5d1d70e","Type":"ContainerDied","Data":"de27f51d97f01c23b3ace53a659d3cfc4ca4aa6881ff38aca2d2df49d45b58a5"}
Jul 15 16:08:01.495606 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.495550    2580 scope.go:117] "RemoveContainer" containerID="cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1"
Jul 15 16:08:01.505969 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.505946    2580 scope.go:117] "RemoveContainer" containerID="cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1"
Jul 15 16:08:01.506281 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:08:01.506260    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1\": container with ID starting with cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1 not found: ID does not exist" containerID="cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1"
Jul 15 16:08:01.506352 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.506293    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1"} err="failed to get container status \"cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1\": rpc error: code = NotFound desc = could not find container \"cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1\": container with ID starting with cc38719a92d4fcedc93225ca006fe17b8f2f34c42e7cb323958ced04ccbdc4f1 not found: ID does not exist"
Jul 15 16:08:01.531249 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.531177    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"]
Jul 15 16:08:01.537706 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.537682    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/auth-test-model-bcb615-kserve-fb5bbdc75-dlp4x"]
Jul 15 16:08:01.767839 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.767809    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" path="/var/lib/kubelet/pods/2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5/volumes"
Jul 15 16:08:01.768214 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:01.768200    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" path="/var/lib/kubelet/pods/61ca1b35-3edc-40e9-8803-f0b3f5d1d70e/volumes"
Jul 15 16:08:03.674209 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:03.674174    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:08:32.577535 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.577487    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"]
Jul 15 16:08:32.578170 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.578149    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" containerName="main"
Jul 15 16:08:32.578259 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.578175    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" containerName="main"
Jul 15 16:08:32.578259 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.578197    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" containerName="main"
Jul 15 16:08:32.578259 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.578206    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" containerName="main"
Jul 15 16:08:32.578355 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.578335    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="61ca1b35-3edc-40e9-8803-f0b3f5d1d70e" containerName="main"
Jul 15 16:08:32.578355 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.578351    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d0d29d7-13c0-4f76-9bbb-7e5b27456ab5" containerName="main"
Jul 15 16:08:32.581890 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.581871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.585810 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.585781    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"default-dockercfg-2462r\""
Jul 15 16:08:32.585941 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.585820    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"sub-test-model-bcb615-kserve-self-signed-certs\""
Jul 15 16:08:32.585941 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.585851    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"kube-root-ca.crt\""
Jul 15 16:08:32.585941 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.585826    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"openshift-service-ca.crt\""
Jul 15 16:08:32.592235 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.592215    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"]
Jul 15 16:08:32.651807 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.651746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-dshm\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.651986 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.651829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcqx6\" (UniqueName: \"kubernetes.io/projected/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-kube-api-access-jcqx6\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.651986 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.651924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-model-cache\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.651986 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.651970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-home\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.652130 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.652001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tmp-dir\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.652130 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.652055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tls-certs\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.752895 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.752862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tls-certs\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753088 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.752928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-dshm\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753088 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.752957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcqx6\" (UniqueName: \"kubernetes.io/projected/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-kube-api-access-jcqx6\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753216 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.753103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-model-cache\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753216 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.753142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-home\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753216 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.753167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tmp-dir\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753519 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.753484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-home\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753519 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.753482    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-model-cache\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.753670 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.753516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tmp-dir\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.755314 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.755286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-dshm\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.755461 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.755443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tls-certs\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.762181 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.762158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcqx6\" (UniqueName: \"kubernetes.io/projected/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-kube-api-access-jcqx6\") pod \"sub-test-model-bcb615-kserve-749d84dccc-kkh5k\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") " pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:32.892052 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:32.891959    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:33.029020 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:33.028992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"]
Jul 15 16:08:33.031503 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:08:33.031474    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod374b9fe6_0c7e_43b4_9652_32cbad3f05c6.slice/crio-c4de0b85143a01f50c8d5a22720d456031be45d588cd26e8556e709104813f46 WatchSource:0}: Error finding container c4de0b85143a01f50c8d5a22720d456031be45d588cd26e8556e709104813f46: Status 404 returned error can't find the container with id c4de0b85143a01f50c8d5a22720d456031be45d588cd26e8556e709104813f46
Jul 15 16:08:33.696643 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:33.696605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k" event={"ID":"374b9fe6-0c7e-43b4-9652-32cbad3f05c6","Type":"ContainerStarted","Data":"1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c"}
Jul 15 16:08:33.696643 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:33.696650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k" event={"ID":"374b9fe6-0c7e-43b4-9652-32cbad3f05c6","Type":"ContainerStarted","Data":"c4de0b85143a01f50c8d5a22720d456031be45d588cd26e8556e709104813f46"}
Jul 15 16:08:33.721729 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:33.721668    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k" podStartSLOduration=1.721648027 podStartE2EDuration="1.721648027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:08:33.716951735 +0000 UTC m=+2638.518411586" watchObservedRunningTime="2026-07-15 16:08:33.721648027 +0000 UTC m=+2638.523107842"
Jul 15 16:08:36.871704 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:36.871661    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:08:42.892847 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:42.892811    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:42.893315 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:42.892863    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:42.905380 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:42.905354    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:08:43.770388 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:08:43.770361    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:09:02.465070 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:02.465034    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:09:20.173446 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:20.173412    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:09:26.847983 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:26.847952    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"]
Jul 15 16:09:35.946567 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:35.946537    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 16:09:35.957382 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:35.957359    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m46hw_9a15276c-947b-43db-abc2-d95c4025e505/ovn-acl-logging/0.log"
Jul 15 16:09:39.571081 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:39.571050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:09:41.868476 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:41.868438    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k" podUID="374b9fe6-0c7e-43b4-9652-32cbad3f05c6" containerName="main" containerID="cri-o://1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c" gracePeriod=45
Jul 15 16:09:42.127976 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.127911    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:09:42.138882 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.138854    2580 generic.go:356] "Generic (PLEG): container finished" podID="374b9fe6-0c7e-43b4-9652-32cbad3f05c6" containerID="1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c" exitCode=0
Jul 15 16:09:42.139048 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.138890    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"
Jul 15 16:09:42.139048 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.138916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k" event={"ID":"374b9fe6-0c7e-43b4-9652-32cbad3f05c6","Type":"ContainerDied","Data":"1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c"}
Jul 15 16:09:42.139048 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.138958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k" event={"ID":"374b9fe6-0c7e-43b4-9652-32cbad3f05c6","Type":"ContainerDied","Data":"c4de0b85143a01f50c8d5a22720d456031be45d588cd26e8556e709104813f46"}
Jul 15 16:09:42.139048 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.138981    2580 scope.go:117] "RemoveContainer" containerID="1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c"
Jul 15 16:09:42.149083 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.149061    2580 scope.go:117] "RemoveContainer" containerID="1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c"
Jul 15 16:09:42.149394 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:09:42.149377    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c\": container with ID starting with 1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c not found: ID does not exist" containerID="1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c"
Jul 15 16:09:42.149471 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.149407    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c"} err="failed to get container status \"1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c\": rpc error: code = NotFound desc = could not find container \"1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c\": container with ID starting with 1eeabf656ade917d8352b173e2bd03c2d85b020f3e3ae1c53a6cce903057bf9c not found: ID does not exist"
Jul 15 16:09:42.203505 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203467    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-dshm\") pod \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") "
Jul 15 16:09:42.203505 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203514    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tmp-dir\") pod \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") "
Jul 15 16:09:42.203714 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcqx6\" (UniqueName: \"kubernetes.io/projected/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-kube-api-access-jcqx6\") pod \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") "
Jul 15 16:09:42.203714 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203604    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tls-certs\") pod \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") "
Jul 15 16:09:42.203714 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-model-cache\") pod \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") "
Jul 15 16:09:42.203903 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203725    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-home\") pod \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\" (UID: \"374b9fe6-0c7e-43b4-9652-32cbad3f05c6\") "
Jul 15 16:09:42.203903 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203866    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "374b9fe6-0c7e-43b4-9652-32cbad3f05c6" (UID: "374b9fe6-0c7e-43b4-9652-32cbad3f05c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:09:42.204012 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.203952    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "374b9fe6-0c7e-43b4-9652-32cbad3f05c6" (UID: "374b9fe6-0c7e-43b4-9652-32cbad3f05c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:09:42.204067 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.204024    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-home" (OuterVolumeSpecName: "home") pod "374b9fe6-0c7e-43b4-9652-32cbad3f05c6" (UID: "374b9fe6-0c7e-43b4-9652-32cbad3f05c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:09:42.204147 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.204133    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:09:42.204187 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.204149    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:09:42.204187 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.204160    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:09:42.205848 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.205821    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "374b9fe6-0c7e-43b4-9652-32cbad3f05c6" (UID: "374b9fe6-0c7e-43b4-9652-32cbad3f05c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:09:42.206234 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.206213    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-kube-api-access-jcqx6" (OuterVolumeSpecName: "kube-api-access-jcqx6") pod "374b9fe6-0c7e-43b4-9652-32cbad3f05c6" (UID: "374b9fe6-0c7e-43b4-9652-32cbad3f05c6"). InnerVolumeSpecName "kube-api-access-jcqx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:09:42.206301 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.206237    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-dshm" (OuterVolumeSpecName: "dshm") pod "374b9fe6-0c7e-43b4-9652-32cbad3f05c6" (UID: "374b9fe6-0c7e-43b4-9652-32cbad3f05c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:09:42.304932 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.304905    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:09:42.304932 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.304930    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcqx6\" (UniqueName: \"kubernetes.io/projected/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-kube-api-access-jcqx6\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:09:42.305116 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.304941    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/374b9fe6-0c7e-43b4-9652-32cbad3f05c6-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:09:42.461528 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.461497    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"]
Jul 15 16:09:42.473784 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:42.468856    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/sub-test-model-bcb615-kserve-749d84dccc-kkh5k"]
Jul 15 16:09:43.767764 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:43.767728    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="374b9fe6-0c7e-43b4-9652-32cbad3f05c6" path="/var/lib/kubelet/pods/374b9fe6-0c7e-43b4-9652-32cbad3f05c6/volumes"
Jul 15 16:09:53.072610 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.072578    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"]
Jul 15 16:09:53.073110 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.073008    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b9fe6-0c7e-43b4-9652-32cbad3f05c6" containerName="main"
Jul 15 16:09:53.073110 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.073020    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b9fe6-0c7e-43b4-9652-32cbad3f05c6" containerName="main"
Jul 15 16:09:53.073110 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.073103    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="374b9fe6-0c7e-43b4-9652-32cbad3f05c6" containerName="main"
Jul 15 16:09:53.076209 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.076193    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.079939 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.079914    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"default-dockercfg-lxlzj\""
Jul 15 16:09:53.080059 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.079938    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"openshift-service-ca.crt\""
Jul 15 16:09:53.080059 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.079916    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"rate-test-model-0e8a6d-kserve-self-signed-certs\""
Jul 15 16:09:53.080059 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.080015    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0e8a6d\"/\"kube-root-ca.crt\""
Jul 15 16:09:53.088311 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.088291    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"]
Jul 15 16:09:53.200293 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.200263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-home\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.200475 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.200296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-tmp-dir\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.200475 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.200328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nt2h\" (UniqueName: \"kubernetes.io/projected/08e11c91-b94b-44bf-9697-02767f57aa6c-kube-api-access-7nt2h\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.200475 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.200390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08e11c91-b94b-44bf-9697-02767f57aa6c-tls-certs\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.200597 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.200545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-dshm\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.200649 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.200593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-model-cache\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.301804 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.301741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-dshm\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.301981 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.301901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-model-cache\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.301981 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.301964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-home\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.302107 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.301984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-tmp-dir\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.302107 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.302015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nt2h\" (UniqueName: \"kubernetes.io/projected/08e11c91-b94b-44bf-9697-02767f57aa6c-kube-api-access-7nt2h\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.302107 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.302064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08e11c91-b94b-44bf-9697-02767f57aa6c-tls-certs\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.302269 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.302249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-model-cache\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.302331 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.302315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-tmp-dir\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.302526 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.302509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-home\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.304145 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.304125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-dshm\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.304408 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.304390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08e11c91-b94b-44bf-9697-02767f57aa6c-tls-certs\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.311636 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.311613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nt2h\" (UniqueName: \"kubernetes.io/projected/08e11c91-b94b-44bf-9697-02767f57aa6c-kube-api-access-7nt2h\") pod \"rate-test-model-0e8a6d-kserve-76df649c9-5vwcc\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") " pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.385255 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.385182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:09:53.520966 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:53.520936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"]
Jul 15 16:09:53.522716 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:09:53.522694    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08e11c91_b94b_44bf_9697_02767f57aa6c.slice/crio-e92efb701bf50f3ef9c16ffab371d018d1d3b06d7fa7959e36179e4d1a3d8372 WatchSource:0}: Error finding container e92efb701bf50f3ef9c16ffab371d018d1d3b06d7fa7959e36179e4d1a3d8372: Status 404 returned error can't find the container with id e92efb701bf50f3ef9c16ffab371d018d1d3b06d7fa7959e36179e4d1a3d8372
Jul 15 16:09:54.214918 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:54.214881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc" event={"ID":"08e11c91-b94b-44bf-9697-02767f57aa6c","Type":"ContainerStarted","Data":"31ed8330af78b172733529742b514257309be468c693df1edc94f355a826105c"}
Jul 15 16:09:54.215298 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:54.214933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc" event={"ID":"08e11c91-b94b-44bf-9697-02767f57aa6c","Type":"ContainerStarted","Data":"e92efb701bf50f3ef9c16ffab371d018d1d3b06d7fa7959e36179e4d1a3d8372"}
Jul 15 16:09:54.234273 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:54.234214    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc" podStartSLOduration=1.234199232 podStartE2EDuration="1.234199232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:09:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:09:54.233571247 +0000 UTC m=+2719.035031067" watchObservedRunningTime="2026-07-15 16:09:54.234199232 +0000 UTC m=+2719.035659057"
Jul 15 16:09:56.866735 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:09:56.866697    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:10:03.385485 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:03.385441    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:10:03.385902 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:03.385503    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:10:03.398272 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:03.398249    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:10:04.284527 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:04.284496    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:10:13.176774 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.176711    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"]
Jul 15 16:10:13.181135 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.181111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.190045 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.189994    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"kube-root-ca.crt\""
Jul 15 16:10:13.190642 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.190097    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"default-dockercfg-2462r\""
Jul 15 16:10:13.190851 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.190189    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"rate-test-model-bcb615-kserve-self-signed-certs\""
Jul 15 16:10:13.190961 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.190730    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bcb615\"/\"openshift-service-ca.crt\""
Jul 15 16:10:13.194995 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.194960    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"]
Jul 15 16:10:13.287283 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.287246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c846ae7d-994c-4fb7-aee0-748a9ad35609-tls-certs\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.287444 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.287342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-tmp-dir\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.287444 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.287382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-476vs\" (UniqueName: \"kubernetes.io/projected/c846ae7d-994c-4fb7-aee0-748a9ad35609-kube-api-access-476vs\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.287444 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.287421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-home\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.287579 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.287489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-dshm\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.287579 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.287523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-model-cache\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.388528 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.388499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c846ae7d-994c-4fb7-aee0-748a9ad35609-tls-certs\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.388698 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.388545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-tmp-dir\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.388698 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.388654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-476vs\" (UniqueName: \"kubernetes.io/projected/c846ae7d-994c-4fb7-aee0-748a9ad35609-kube-api-access-476vs\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.388835 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.388720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-home\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.388835 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.388763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-dshm\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.388835 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.388788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-model-cache\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.388987 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.388880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-tmp-dir\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.389124 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.389093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-home\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.389247 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.389169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-model-cache\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.390998 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.390968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-dshm\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.391124 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.391109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c846ae7d-994c-4fb7-aee0-748a9ad35609-tls-certs\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.397026 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.397007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-476vs\" (UniqueName: \"kubernetes.io/projected/c846ae7d-994c-4fb7-aee0-748a9ad35609-kube-api-access-476vs\") pod \"rate-test-model-bcb615-kserve-f657c854c-q7bbr\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") " pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.499415 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.499344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:13.641828 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:13.641803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"]
Jul 15 16:10:13.643454 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:10:13.643427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc846ae7d_994c_4fb7_aee0_748a9ad35609.slice/crio-74a0a28ef63baaafaef441078b6642560f6982c89b3b45a4b6e5822a97a1a44f WatchSource:0}: Error finding container 74a0a28ef63baaafaef441078b6642560f6982c89b3b45a4b6e5822a97a1a44f: Status 404 returned error can't find the container with id 74a0a28ef63baaafaef441078b6642560f6982c89b3b45a4b6e5822a97a1a44f
Jul 15 16:10:14.336126 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:14.336090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr" event={"ID":"c846ae7d-994c-4fb7-aee0-748a9ad35609","Type":"ContainerStarted","Data":"ce8521cce9ff3d8694be6b5258ec427e09c76ca7e13d8f80dea766ee47f35760"}
Jul 15 16:10:14.336510 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:14.336135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr" event={"ID":"c846ae7d-994c-4fb7-aee0-748a9ad35609","Type":"ContainerStarted","Data":"74a0a28ef63baaafaef441078b6642560f6982c89b3b45a4b6e5822a97a1a44f"}
Jul 15 16:10:14.353917 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:14.353864    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr" podStartSLOduration=1.353849114 podStartE2EDuration="1.353849114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:10:14.352819147 +0000 UTC m=+2739.154278963" watchObservedRunningTime="2026-07-15 16:10:14.353849114 +0000 UTC m=+2739.155308928"
Jul 15 16:10:17.187797 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:17.187739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:10:23.499786 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:23.499728    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:23.500278 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:23.499817    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:23.513263 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:23.513237    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:24.408983 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:24.408954    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:10:37.768894 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:37.768859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:10:52.970167 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:10:52.970122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:11:00.496684 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:00.496612    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"]
Jul 15 16:11:01.457381 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:01.457348    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"]
Jul 15 16:11:06.969744 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:06.969706    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:11:15.517134 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.517070    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc" podUID="08e11c91-b94b-44bf-9697-02767f57aa6c" containerName="main" containerID="cri-o://31ed8330af78b172733529742b514257309be468c693df1edc94f355a826105c" gracePeriod=45
Jul 15 16:11:15.714899 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.714867    2580 generic.go:356] "Generic (PLEG): container finished" podID="08e11c91-b94b-44bf-9697-02767f57aa6c" containerID="31ed8330af78b172733529742b514257309be468c693df1edc94f355a826105c" exitCode=0
Jul 15 16:11:15.715060 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.714927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc" event={"ID":"08e11c91-b94b-44bf-9697-02767f57aa6c","Type":"ContainerDied","Data":"31ed8330af78b172733529742b514257309be468c693df1edc94f355a826105c"}
Jul 15 16:11:15.775285 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.775232    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:11:15.894787 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.894740    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-dshm\") pod \"08e11c91-b94b-44bf-9697-02767f57aa6c\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") "
Jul 15 16:11:15.894952 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.894816    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-model-cache\") pod \"08e11c91-b94b-44bf-9697-02767f57aa6c\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") "
Jul 15 16:11:15.894952 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.894834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-home\") pod \"08e11c91-b94b-44bf-9697-02767f57aa6c\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") "
Jul 15 16:11:15.894952 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.894910    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08e11c91-b94b-44bf-9697-02767f57aa6c-tls-certs\") pod \"08e11c91-b94b-44bf-9697-02767f57aa6c\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") "
Jul 15 16:11:15.894952 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.894944    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-tmp-dir\") pod \"08e11c91-b94b-44bf-9697-02767f57aa6c\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") "
Jul 15 16:11:15.895168 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.894996    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nt2h\" (UniqueName: \"kubernetes.io/projected/08e11c91-b94b-44bf-9697-02767f57aa6c-kube-api-access-7nt2h\") pod \"08e11c91-b94b-44bf-9697-02767f57aa6c\" (UID: \"08e11c91-b94b-44bf-9697-02767f57aa6c\") "
Jul 15 16:11:15.895168 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.895094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-model-cache" (OuterVolumeSpecName: "model-cache") pod "08e11c91-b94b-44bf-9697-02767f57aa6c" (UID: "08e11c91-b94b-44bf-9697-02767f57aa6c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:15.895290 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.895195    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-home" (OuterVolumeSpecName: "home") pod "08e11c91-b94b-44bf-9697-02767f57aa6c" (UID: "08e11c91-b94b-44bf-9697-02767f57aa6c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:15.895370 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.895352    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:15.895437 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.895361    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "08e11c91-b94b-44bf-9697-02767f57aa6c" (UID: "08e11c91-b94b-44bf-9697-02767f57aa6c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:15.895437 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.895373    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:15.897050 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.897021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e11c91-b94b-44bf-9697-02767f57aa6c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "08e11c91-b94b-44bf-9697-02767f57aa6c" (UID: "08e11c91-b94b-44bf-9697-02767f57aa6c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:11:15.897160 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.897062    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-dshm" (OuterVolumeSpecName: "dshm") pod "08e11c91-b94b-44bf-9697-02767f57aa6c" (UID: "08e11c91-b94b-44bf-9697-02767f57aa6c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:15.897160 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.897079    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e11c91-b94b-44bf-9697-02767f57aa6c-kube-api-access-7nt2h" (OuterVolumeSpecName: "kube-api-access-7nt2h") pod "08e11c91-b94b-44bf-9697-02767f57aa6c" (UID: "08e11c91-b94b-44bf-9697-02767f57aa6c"). InnerVolumeSpecName "kube-api-access-7nt2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:11:15.996547 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.996513    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08e11c91-b94b-44bf-9697-02767f57aa6c-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:15.996716 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.996565    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:15.996716 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.996578    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nt2h\" (UniqueName: \"kubernetes.io/projected/08e11c91-b94b-44bf-9697-02767f57aa6c-kube-api-access-7nt2h\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:15.996716 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:15.996590    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08e11c91-b94b-44bf-9697-02767f57aa6c-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:16.477635 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.477592    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr" podUID="c846ae7d-994c-4fb7-aee0-748a9ad35609" containerName="main" containerID="cri-o://ce8521cce9ff3d8694be6b5258ec427e09c76ca7e13d8f80dea766ee47f35760" gracePeriod=45
Jul 15 16:11:16.722916 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.722786    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"
Jul 15 16:11:16.722916 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.722790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc" event={"ID":"08e11c91-b94b-44bf-9697-02767f57aa6c","Type":"ContainerDied","Data":"e92efb701bf50f3ef9c16ffab371d018d1d3b06d7fa7959e36179e4d1a3d8372"}
Jul 15 16:11:16.722916 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.722903    2580 scope.go:117] "RemoveContainer" containerID="31ed8330af78b172733529742b514257309be468c693df1edc94f355a826105c"
Jul 15 16:11:16.724888 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.724864    2580 generic.go:356] "Generic (PLEG): container finished" podID="c846ae7d-994c-4fb7-aee0-748a9ad35609" containerID="ce8521cce9ff3d8694be6b5258ec427e09c76ca7e13d8f80dea766ee47f35760" exitCode=0
Jul 15 16:11:16.725046 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.724933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr" event={"ID":"c846ae7d-994c-4fb7-aee0-748a9ad35609","Type":"ContainerDied","Data":"ce8521cce9ff3d8694be6b5258ec427e09c76ca7e13d8f80dea766ee47f35760"}
Jul 15 16:11:16.743518 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.743502    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:11:16.756529 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.756507    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"]
Jul 15 16:11:16.765205 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.765182    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0e8a6d/rate-test-model-0e8a6d-kserve-76df649c9-5vwcc"]
Jul 15 16:11:16.904223 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-model-cache\") pod \"c846ae7d-994c-4fb7-aee0-748a9ad35609\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") "
Jul 15 16:11:16.904385 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904308    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-dshm\") pod \"c846ae7d-994c-4fb7-aee0-748a9ad35609\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") "
Jul 15 16:11:16.904385 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-home\") pod \"c846ae7d-994c-4fb7-aee0-748a9ad35609\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") "
Jul 15 16:11:16.904459 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904384    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c846ae7d-994c-4fb7-aee0-748a9ad35609-tls-certs\") pod \"c846ae7d-994c-4fb7-aee0-748a9ad35609\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") "
Jul 15 16:11:16.904459 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904408    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-tmp-dir\") pod \"c846ae7d-994c-4fb7-aee0-748a9ad35609\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") "
Jul 15 16:11:16.904459 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-476vs\" (UniqueName: \"kubernetes.io/projected/c846ae7d-994c-4fb7-aee0-748a9ad35609-kube-api-access-476vs\") pod \"c846ae7d-994c-4fb7-aee0-748a9ad35609\" (UID: \"c846ae7d-994c-4fb7-aee0-748a9ad35609\") "
Jul 15 16:11:16.904606 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904471    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-model-cache" (OuterVolumeSpecName: "model-cache") pod "c846ae7d-994c-4fb7-aee0-748a9ad35609" (UID: "c846ae7d-994c-4fb7-aee0-748a9ad35609"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:16.904661 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904602    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-home" (OuterVolumeSpecName: "home") pod "c846ae7d-994c-4fb7-aee0-748a9ad35609" (UID: "c846ae7d-994c-4fb7-aee0-748a9ad35609"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:16.904789 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904730    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c846ae7d-994c-4fb7-aee0-748a9ad35609" (UID: "c846ae7d-994c-4fb7-aee0-748a9ad35609"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:16.904885 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904839    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-home\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:16.904885 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.904854    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-model-cache\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:16.906799 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.906775    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c846ae7d-994c-4fb7-aee0-748a9ad35609-kube-api-access-476vs" (OuterVolumeSpecName: "kube-api-access-476vs") pod "c846ae7d-994c-4fb7-aee0-748a9ad35609" (UID: "c846ae7d-994c-4fb7-aee0-748a9ad35609"). InnerVolumeSpecName "kube-api-access-476vs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:11:16.906922 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.906893    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-dshm" (OuterVolumeSpecName: "dshm") pod "c846ae7d-994c-4fb7-aee0-748a9ad35609" (UID: "c846ae7d-994c-4fb7-aee0-748a9ad35609"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:11:16.906986 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:16.906929    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c846ae7d-994c-4fb7-aee0-748a9ad35609-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c846ae7d-994c-4fb7-aee0-748a9ad35609" (UID: "c846ae7d-994c-4fb7-aee0-748a9ad35609"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:11:17.005794 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.005707    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-dshm\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:17.005794 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.005733    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c846ae7d-994c-4fb7-aee0-748a9ad35609-tls-certs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:17.005794 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.005742    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c846ae7d-994c-4fb7-aee0-748a9ad35609-tmp-dir\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:17.005794 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.005767    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-476vs\" (UniqueName: \"kubernetes.io/projected/c846ae7d-994c-4fb7-aee0-748a9ad35609-kube-api-access-476vs\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:17.150522 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.150488    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140545741309248"]
Jul 15 16:11:17.150951 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.150936    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e11c91-b94b-44bf-9697-02767f57aa6c" containerName="main"
Jul 15 16:11:17.151004 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.150953    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e11c91-b94b-44bf-9697-02767f57aa6c" containerName="main"
Jul 15 16:11:17.151004 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.150965    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c846ae7d-994c-4fb7-aee0-748a9ad35609" containerName="main"
Jul 15 16:11:17.151004 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.150970    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c846ae7d-994c-4fb7-aee0-748a9ad35609" containerName="main"
Jul 15 16:11:17.151101 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.151034    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e11c91-b94b-44bf-9697-02767f57aa6c" containerName="main"
Jul 15 16:11:17.151101 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.151045    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c846ae7d-994c-4fb7-aee0-748a9ad35609" containerName="main"
Jul 15 16:11:17.154079 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.154063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:17.157440 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.157419    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-br7zr\""
Jul 15 16:11:17.159082 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.159061    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140545741309248"]
Jul 15 16:11:17.309567 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.309477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxv2c\" (UniqueName: \"kubernetes.io/projected/cb6d363d-7fef-49d9-bec2-9bf123fa92ab-kube-api-access-gxv2c\") pod \"test-curl-7426-140545741309248\" (UID: \"cb6d363d-7fef-49d9-bec2-9bf123fa92ab\") " pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:17.410707 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.410668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxv2c\" (UniqueName: \"kubernetes.io/projected/cb6d363d-7fef-49d9-bec2-9bf123fa92ab-kube-api-access-gxv2c\") pod \"test-curl-7426-140545741309248\" (UID: \"cb6d363d-7fef-49d9-bec2-9bf123fa92ab\") " pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:17.418978 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.418950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxv2c\" (UniqueName: \"kubernetes.io/projected/cb6d363d-7fef-49d9-bec2-9bf123fa92ab-kube-api-access-gxv2c\") pod \"test-curl-7426-140545741309248\" (UID: \"cb6d363d-7fef-49d9-bec2-9bf123fa92ab\") " pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:17.464798 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.464772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:17.600326 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.600281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140545741309248"]
Jul 15 16:11:17.603154 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:11:17.603124    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb6d363d_7fef_49d9_bec2_9bf123fa92ab.slice/crio-294ffcae1b4256012751a1ec8a84dfc2cee687a2aed6124105e69f35dc46d815 WatchSource:0}: Error finding container 294ffcae1b4256012751a1ec8a84dfc2cee687a2aed6124105e69f35dc46d815: Status 404 returned error can't find the container with id 294ffcae1b4256012751a1ec8a84dfc2cee687a2aed6124105e69f35dc46d815
Jul 15 16:11:17.604619 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.604598    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 16:11:17.733102 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.733074    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"
Jul 15 16:11:17.733525 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.733114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr" event={"ID":"c846ae7d-994c-4fb7-aee0-748a9ad35609","Type":"ContainerDied","Data":"74a0a28ef63baaafaef441078b6642560f6982c89b3b45a4b6e5822a97a1a44f"}
Jul 15 16:11:17.733525 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.733165    2580 scope.go:117] "RemoveContainer" containerID="ce8521cce9ff3d8694be6b5258ec427e09c76ca7e13d8f80dea766ee47f35760"
Jul 15 16:11:17.735542 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.735515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140545741309248" event={"ID":"cb6d363d-7fef-49d9-bec2-9bf123fa92ab","Type":"ContainerStarted","Data":"294ffcae1b4256012751a1ec8a84dfc2cee687a2aed6124105e69f35dc46d815"}
Jul 15 16:11:17.756543 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.756519    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"]
Jul 15 16:11:17.773764 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.773726    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08e11c91-b94b-44bf-9697-02767f57aa6c" path="/var/lib/kubelet/pods/08e11c91-b94b-44bf-9697-02767f57aa6c/volumes"
Jul 15 16:11:17.774027 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:17.774013    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bcb615/rate-test-model-bcb615-kserve-f657c854c-q7bbr"]
Jul 15 16:11:18.744785 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:18.744731    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb6d363d-7fef-49d9-bec2-9bf123fa92ab" containerID="1d1a0a59b2671f30a0416a88cf103f4b4285dd2280d2aae1a674766eec78b2e5" exitCode=0
Jul 15 16:11:18.745166 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:18.744784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140545741309248" event={"ID":"cb6d363d-7fef-49d9-bec2-9bf123fa92ab","Type":"ContainerDied","Data":"1d1a0a59b2671f30a0416a88cf103f4b4285dd2280d2aae1a674766eec78b2e5"}
Jul 15 16:11:19.772776 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:19.772718    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c846ae7d-994c-4fb7-aee0-748a9ad35609" path="/var/lib/kubelet/pods/c846ae7d-994c-4fb7-aee0-748a9ad35609/volumes"
Jul 15 16:11:19.891079 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:19.891058    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:19.927767 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:19.927732    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140545741309248_cb6d363d-7fef-49d9-bec2-9bf123fa92ab/test-curl-7426-140545741309248/0.log"
Jul 15 16:11:19.950801 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:19.950770    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140545741309248"]
Jul 15 16:11:19.954513 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:19.954483    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140545741309248"]
Jul 15 16:11:20.035739 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.035647    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxv2c\" (UniqueName: \"kubernetes.io/projected/cb6d363d-7fef-49d9-bec2-9bf123fa92ab-kube-api-access-gxv2c\") pod \"cb6d363d-7fef-49d9-bec2-9bf123fa92ab\" (UID: \"cb6d363d-7fef-49d9-bec2-9bf123fa92ab\") "
Jul 15 16:11:20.037874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.037854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb6d363d-7fef-49d9-bec2-9bf123fa92ab-kube-api-access-gxv2c" (OuterVolumeSpecName: "kube-api-access-gxv2c") pod "cb6d363d-7fef-49d9-bec2-9bf123fa92ab" (UID: "cb6d363d-7fef-49d9-bec2-9bf123fa92ab"). InnerVolumeSpecName "kube-api-access-gxv2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:11:20.137142 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.137107    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxv2c\" (UniqueName: \"kubernetes.io/projected/cb6d363d-7fef-49d9-bec2-9bf123fa92ab-kube-api-access-gxv2c\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:20.157488 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.157458    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"]
Jul 15 16:11:20.157946 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.157929    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb6d363d-7fef-49d9-bec2-9bf123fa92ab" containerName="test-curl-7426-140545741309248"
Jul 15 16:11:20.157946 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.157946    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb6d363d-7fef-49d9-bec2-9bf123fa92ab" containerName="test-curl-7426-140545741309248"
Jul 15 16:11:20.158117 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.158033    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb6d363d-7fef-49d9-bec2-9bf123fa92ab" containerName="test-curl-7426-140545741309248"
Jul 15 16:11:20.161271 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.161254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"
Jul 15 16:11:20.163929 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.163910    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 16:11:20.170267 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.169839    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"]
Jul 15 16:11:20.338769 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.338708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x\" (UID: \"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"
Jul 15 16:11:20.439627 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.439586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x\" (UID: \"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"
Jul 15 16:11:20.440257 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.440238    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x\" (UID: \"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"
Jul 15 16:11:20.473527 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.473495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"
Jul 15 16:11:20.612223 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.612191    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"]
Jul 15 16:11:20.613573 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:11:20.613548    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1c6a7f_2bb2_4e0e_9843_c9faabbfb6bd.slice/crio-bea78039008d2baffac12a7ee62950cf9187eb147ad0b734934b4dc1bb74cf2e WatchSource:0}: Error finding container bea78039008d2baffac12a7ee62950cf9187eb147ad0b734934b4dc1bb74cf2e: Status 404 returned error can't find the container with id bea78039008d2baffac12a7ee62950cf9187eb147ad0b734934b4dc1bb74cf2e
Jul 15 16:11:20.758958 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.758919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x" event={"ID":"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd","Type":"ContainerStarted","Data":"bea78039008d2baffac12a7ee62950cf9187eb147ad0b734934b4dc1bb74cf2e"}
Jul 15 16:11:20.760111 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.760080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:20.760111 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:20.760098    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="294ffcae1b4256012751a1ec8a84dfc2cee687a2aed6124105e69f35dc46d815"
Jul 15 16:11:20.770370 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:11:20.770344    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140545741309248\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object" podUID="cb6d363d-7fef-49d9-bec2-9bf123fa92ab" pod="opendatahub/test-curl-7426-140545741309248"
Jul 15 16:11:21.767467 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:21.767379    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb6d363d-7fef-49d9-bec2-9bf123fa92ab" path="/var/lib/kubelet/pods/cb6d363d-7fef-49d9-bec2-9bf123fa92ab/volumes"
Jul 15 16:11:21.767918 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:21.767870    2580 generic.go:356] "Generic (PLEG): container finished" podID="da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd" containerID="624172835d9a028ca23f8305ca7665c91dc4a127d6b021ae6abb5d29d56995b6" exitCode=0
Jul 15 16:11:21.767962 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:21.767916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x" event={"ID":"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd","Type":"ContainerDied","Data":"624172835d9a028ca23f8305ca7665c91dc4a127d6b021ae6abb5d29d56995b6"}
Jul 15 16:11:22.905196 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:22.905171    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"
Jul 15 16:11:22.961676 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:22.961645    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd-maas-api-service-ca\") pod \"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd\" (UID: \"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd\") "
Jul 15 16:11:22.961996 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:22.961975    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd" (UID: "da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:11:23.063219 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:23.063139    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:23.781212 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:23.781189    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x"
Jul 15 16:11:23.781326 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:23.781193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x" event={"ID":"da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd","Type":"ContainerDied","Data":"bea78039008d2baffac12a7ee62950cf9187eb147ad0b734934b4dc1bb74cf2e"}
Jul 15 16:11:23.781326 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:23.781292    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bea78039008d2baffac12a7ee62950cf9187eb147ad0b734934b4dc1bb74cf2e"
Jul 15 16:11:30.622036 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.621992    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"]
Jul 15 16:11:30.622530 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.622255    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng" podUID="8d58aec0-ad57-4f0e-a8f3-7af92192b434" containerName="maas-api" containerID="cri-o://931b804debebb4aa92a879826a67b675d699b04cbd8862a91a08d13c91aba7bd" gracePeriod=30
Jul 15 16:11:30.829145 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.829112    2580 generic.go:356] "Generic (PLEG): container finished" podID="8d58aec0-ad57-4f0e-a8f3-7af92192b434" containerID="931b804debebb4aa92a879826a67b675d699b04cbd8862a91a08d13c91aba7bd" exitCode=0
Jul 15 16:11:30.829314 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.829150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng" event={"ID":"8d58aec0-ad57-4f0e-a8f3-7af92192b434","Type":"ContainerDied","Data":"931b804debebb4aa92a879826a67b675d699b04cbd8862a91a08d13c91aba7bd"}
Jul 15 16:11:30.870517 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.870492    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:11:30.932174 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.932096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mlgf\" (UniqueName: \"kubernetes.io/projected/8d58aec0-ad57-4f0e-a8f3-7af92192b434-kube-api-access-9mlgf\") pod \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") "
Jul 15 16:11:30.932174 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.932172    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d58aec0-ad57-4f0e-a8f3-7af92192b434-openshift-service-ca\") pod \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") "
Jul 15 16:11:30.932358 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.932226    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d58aec0-ad57-4f0e-a8f3-7af92192b434-maas-api-tls\") pod \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\" (UID: \"8d58aec0-ad57-4f0e-a8f3-7af92192b434\") "
Jul 15 16:11:30.932539 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.932510    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d58aec0-ad57-4f0e-a8f3-7af92192b434-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d58aec0-ad57-4f0e-a8f3-7af92192b434" (UID: "8d58aec0-ad57-4f0e-a8f3-7af92192b434"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:11:30.934172 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.934145    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d58aec0-ad57-4f0e-a8f3-7af92192b434-kube-api-access-9mlgf" (OuterVolumeSpecName: "kube-api-access-9mlgf") pod "8d58aec0-ad57-4f0e-a8f3-7af92192b434" (UID: "8d58aec0-ad57-4f0e-a8f3-7af92192b434"). InnerVolumeSpecName "kube-api-access-9mlgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:11:30.934280 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:30.934256    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d58aec0-ad57-4f0e-a8f3-7af92192b434-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d58aec0-ad57-4f0e-a8f3-7af92192b434" (UID: "8d58aec0-ad57-4f0e-a8f3-7af92192b434"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:11:31.033164 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.033130    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d58aec0-ad57-4f0e-a8f3-7af92192b434-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:31.033164 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.033158    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d58aec0-ad57-4f0e-a8f3-7af92192b434-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:31.033164 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.033168    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mlgf\" (UniqueName: \"kubernetes.io/projected/8d58aec0-ad57-4f0e-a8f3-7af92192b434-kube-api-access-9mlgf\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:31.836965 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.836934    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"
Jul 15 16:11:31.837400 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.836935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng" event={"ID":"8d58aec0-ad57-4f0e-a8f3-7af92192b434","Type":"ContainerDied","Data":"77dfb2d2f3e6bc2993c67c3c5915bd41653b58c567e24b658ab339a31e3484e2"}
Jul 15 16:11:31.837400 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.837080    2580 scope.go:117] "RemoveContainer" containerID="931b804debebb4aa92a879826a67b675d699b04cbd8862a91a08d13c91aba7bd"
Jul 15 16:11:31.862589 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.862565    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"]
Jul 15 16:11:31.868010 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:31.867989    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0e8a6d-76458978d5-x2lng"]
Jul 15 16:11:33.768288 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:33.768249    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d58aec0-ad57-4f0e-a8f3-7af92192b434" path="/var/lib/kubelet/pods/8d58aec0-ad57-4f0e-a8f3-7af92192b434/volumes"
Jul 15 16:11:34.265456 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:34.265418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:11:46.292874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.289628    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"]
Jul 15 16:11:46.292874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.290554    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d58aec0-ad57-4f0e-a8f3-7af92192b434" containerName="maas-api"
Jul 15 16:11:46.292874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.290577    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d58aec0-ad57-4f0e-a8f3-7af92192b434" containerName="maas-api"
Jul 15 16:11:46.292874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.290611    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd" containerName="revoke-keys"
Jul 15 16:11:46.292874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.290620    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd" containerName="revoke-keys"
Jul 15 16:11:46.292874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.290816    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d58aec0-ad57-4f0e-a8f3-7af92192b434" containerName="maas-api"
Jul 15 16:11:46.292874 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.290834    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd" containerName="revoke-keys"
Jul 15 16:11:46.295089 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.295061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"
Jul 15 16:11:46.297855 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.297832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tcg2l\""
Jul 15 16:11:46.298301 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.298273    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"]
Jul 15 16:11:46.381066 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.381032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e64f6-5e68-40fe-9dea-49abc8da23c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft\" (UID: \"fc8e64f6-5e68-40fe-9dea-49abc8da23c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"
Jul 15 16:11:46.482038 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.481993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e64f6-5e68-40fe-9dea-49abc8da23c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft\" (UID: \"fc8e64f6-5e68-40fe-9dea-49abc8da23c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"
Jul 15 16:11:46.482597 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.482576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e64f6-5e68-40fe-9dea-49abc8da23c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft\" (UID: \"fc8e64f6-5e68-40fe-9dea-49abc8da23c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"
Jul 15 16:11:46.608251 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.608208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"
Jul 15 16:11:46.751046 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.751016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"]
Jul 15 16:11:46.752787 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:11:46.752744    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc8e64f6_5e68_40fe_9dea_49abc8da23c5.slice/crio-16a8a0432e1da73c7060208730da8aefa013c919d5b76bb14271ef35459cae5c WatchSource:0}: Error finding container 16a8a0432e1da73c7060208730da8aefa013c919d5b76bb14271ef35459cae5c: Status 404 returned error can't find the container with id 16a8a0432e1da73c7060208730da8aefa013c919d5b76bb14271ef35459cae5c
Jul 15 16:11:46.937690 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:46.937658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft" event={"ID":"fc8e64f6-5e68-40fe-9dea-49abc8da23c5","Type":"ContainerStarted","Data":"16a8a0432e1da73c7060208730da8aefa013c919d5b76bb14271ef35459cae5c"}
Jul 15 16:11:47.945023 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:47.944986    2580 generic.go:356] "Generic (PLEG): container finished" podID="fc8e64f6-5e68-40fe-9dea-49abc8da23c5" containerID="18f9aebfefa95912087483c0d8030239d045e0f27352b4114b8c8b7db67171a5" exitCode=0
Jul 15 16:11:47.945485 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:47.945073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft" event={"ID":"fc8e64f6-5e68-40fe-9dea-49abc8da23c5","Type":"ContainerDied","Data":"18f9aebfefa95912087483c0d8030239d045e0f27352b4114b8c8b7db67171a5"}
Jul 15 16:11:49.086678 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:49.086655    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"
Jul 15 16:11:49.208398 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:49.208315    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e64f6-5e68-40fe-9dea-49abc8da23c5-maas-api-service-ca\") pod \"fc8e64f6-5e68-40fe-9dea-49abc8da23c5\" (UID: \"fc8e64f6-5e68-40fe-9dea-49abc8da23c5\") "
Jul 15 16:11:49.208696 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:49.208672    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc8e64f6-5e68-40fe-9dea-49abc8da23c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc8e64f6-5e68-40fe-9dea-49abc8da23c5" (UID: "fc8e64f6-5e68-40fe-9dea-49abc8da23c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:11:49.309496 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:49.309465    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8e64f6-5e68-40fe-9dea-49abc8da23c5-maas-api-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:49.958394 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:49.958356    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft"
Jul 15 16:11:49.958581 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:49.958394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft" event={"ID":"fc8e64f6-5e68-40fe-9dea-49abc8da23c5","Type":"ContainerDied","Data":"16a8a0432e1da73c7060208730da8aefa013c919d5b76bb14271ef35459cae5c"}
Jul 15 16:11:49.958581 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:49.958442    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16a8a0432e1da73c7060208730da8aefa013c919d5b76bb14271ef35459cae5c"
Jul 15 16:11:56.739767 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:56.739717    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"]
Jul 15 16:11:56.740290 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:56.740007    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp" podUID="33ebe502-ab66-4481-b216-7a405e57ba0c" containerName="maas-api" containerID="cri-o://47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a" gracePeriod=30
Jul 15 16:11:56.994125 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:56.994069    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:11:57.002224 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.002198    2580 generic.go:356] "Generic (PLEG): container finished" podID="33ebe502-ab66-4481-b216-7a405e57ba0c" containerID="47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a" exitCode=0
Jul 15 16:11:57.002348 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.002234    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"
Jul 15 16:11:57.002348 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.002237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp" event={"ID":"33ebe502-ab66-4481-b216-7a405e57ba0c","Type":"ContainerDied","Data":"47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a"}
Jul 15 16:11:57.002467 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.002389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp" event={"ID":"33ebe502-ab66-4481-b216-7a405e57ba0c","Type":"ContainerDied","Data":"b8f80b0cf91c842262e3f05afe0003a12146da6fbdd95292c4f6d71047c2323f"}
Jul 15 16:11:57.002467 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.002429    2580 scope.go:117] "RemoveContainer" containerID="47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a"
Jul 15 16:11:57.012600 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.012580    2580 scope.go:117] "RemoveContainer" containerID="47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a"
Jul 15 16:11:57.013078 ip-10-0-135-125 kubenswrapper[2580]: E0715 16:11:57.012998    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a\": container with ID starting with 47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a not found: ID does not exist" containerID="47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a"
Jul 15 16:11:57.013078 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.013051    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a"} err="failed to get container status \"47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a\": rpc error: code = NotFound desc = could not find container \"47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a\": container with ID starting with 47aa7022ee24c56676183502d0b59714af8cd9d58fe1976b67f72bd7cf45fd0a not found: ID does not exist"
Jul 15 16:11:57.084676 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.084649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls\") pod \"33ebe502-ab66-4481-b216-7a405e57ba0c\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") "
Jul 15 16:11:57.084869 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.084738    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpss6\" (UniqueName: \"kubernetes.io/projected/33ebe502-ab66-4481-b216-7a405e57ba0c-kube-api-access-fpss6\") pod \"33ebe502-ab66-4481-b216-7a405e57ba0c\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") "
Jul 15 16:11:57.084869 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.084805    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ebe502-ab66-4481-b216-7a405e57ba0c-openshift-service-ca\") pod \"33ebe502-ab66-4481-b216-7a405e57ba0c\" (UID: \"33ebe502-ab66-4481-b216-7a405e57ba0c\") "
Jul 15 16:11:57.085156 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.085135    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33ebe502-ab66-4481-b216-7a405e57ba0c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33ebe502-ab66-4481-b216-7a405e57ba0c" (UID: "33ebe502-ab66-4481-b216-7a405e57ba0c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:11:57.086780 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.086738    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33ebe502-ab66-4481-b216-7a405e57ba0c" (UID: "33ebe502-ab66-4481-b216-7a405e57ba0c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:11:57.086880 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.086821    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ebe502-ab66-4481-b216-7a405e57ba0c-kube-api-access-fpss6" (OuterVolumeSpecName: "kube-api-access-fpss6") pod "33ebe502-ab66-4481-b216-7a405e57ba0c" (UID: "33ebe502-ab66-4481-b216-7a405e57ba0c"). InnerVolumeSpecName "kube-api-access-fpss6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:11:57.186545 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.186519    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ebe502-ab66-4481-b216-7a405e57ba0c-maas-api-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:57.186545 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.186542    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpss6\" (UniqueName: \"kubernetes.io/projected/33ebe502-ab66-4481-b216-7a405e57ba0c-kube-api-access-fpss6\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:57.186719 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.186553    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ebe502-ab66-4481-b216-7a405e57ba0c-openshift-service-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
Jul 15 16:11:57.325636 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.325599    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"]
Jul 15 16:11:57.330871 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.330845    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bcb615-8478cc96d9-v2wvp"]
Jul 15 16:11:57.767289 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:57.767256    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33ebe502-ab66-4481-b216-7a405e57ba0c" path="/var/lib/kubelet/pods/33ebe502-ab66-4481-b216-7a405e57ba0c/volumes"
Jul 15 16:11:59.965620 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:11:59.965585    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6hw2j"]
Jul 15 16:12:18.132649 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:18.132620    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ktlm2_b8330683-b064-4759-8568-4ebf931d0c21/manager/0.log"
Jul 15 16:12:18.891107 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:18.891076    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-997df6bcb-twxtg_f42580a0-134b-4913-9b8a-61f3b6a61485/copy-manifests/0.log"
Jul 15 16:12:18.904318 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:18.904291    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-997df6bcb-twxtg_f42580a0-134b-4913-9b8a-61f3b6a61485/manager/0.log"
Jul 15 16:12:19.410120 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:19.410091    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735505-vcp6f_740fa25f-39ce-4af9-bc29-dcc5b74c2048/cleanup/0.log"
Jul 15 16:12:19.524831 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:19.524802    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735520-57588_4360de3e-6110-48a2-9763-858e3987a591/cleanup/0.log"
Jul 15 16:12:19.641375 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:19.641348    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-38132c4d-hq4sq_3ba005b4-cf56-43c4-a60a-678d8518deed/revoke-keys/0.log"
Jul 15 16:12:19.763846 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:19.763735    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-c020b6bf-cfsn7_2ac752e5-7849-4ecd-8100-2a8713888c70/revoke-keys/0.log"
Jul 15 16:12:19.882003 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:19.881977    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-097eff-j4rp8_5aa0d377-0472-4814-b61d-066ac5c89f97/revoke-keys/0.log"
Jul 15 16:12:20.008688 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.008656    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-552a11-bx4vx_554a0273-ac6f-40bd-9f1d-904b252fddc2/revoke-keys/0.log"
Jul 15 16:12:20.121393 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.121367    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-4d69cf00-7hqh9_ebd351e5-5d6a-4fac-8f40-706ad71f5da9/revoke-keys/0.log"
Jul 15 16:12:20.343666 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.343637    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-89ce7e-b6m2p_a5c50ad3-f7a4-4fd6-be3b-7341481ddc79/revoke-keys/0.log"
Jul 15 16:12:20.461401 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.461326    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-c7b34001-nd7ft_aeffbf26-85a6-495d-a86f-ea91ff3bbf6c/revoke-keys/0.log"
Jul 15 16:12:20.575590 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.575562    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-7258970a-8dhkj_9ce1659b-0c51-4dc1-a33d-55c13047134c/revoke-keys/0.log"
Jul 15 16:12:20.690097 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.690067    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-fb1842df-j474n_12129e48-ab0f-443b-8ef6-0ac401ad6c57/revoke-keys/0.log"
Jul 15 16:12:20.801470 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.801391    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-0e8a6d-m8g9x_da1c6a7f-2bb2-4e0e-9843-c9faabbfb6bd/revoke-keys/0.log"
Jul 15 16:12:20.928419 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:20.928388    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-bcb615-jhgft_fc8e64f6-5e68-40fe-9dea-49abc8da23c5/revoke-keys/0.log"
Jul 15 16:12:21.861827 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:21.861800    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v_a42fccbc-1d50-4ae0-91a2-763b44861dd9/util/0.log"
Jul 15 16:12:21.872619 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:21.872598    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v_a42fccbc-1d50-4ae0-91a2-763b44861dd9/pull/0.log"
Jul 15 16:12:21.880380 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:21.880358    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v_a42fccbc-1d50-4ae0-91a2-763b44861dd9/extract/0.log"
Jul 15 16:12:22.005264 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.005234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc_d158794c-7d7d-42c8-8364-72aac59b575b/util/0.log"
Jul 15 16:12:22.013950 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.013922    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc_d158794c-7d7d-42c8-8364-72aac59b575b/pull/0.log"
Jul 15 16:12:22.020951 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.020930    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc_d158794c-7d7d-42c8-8364-72aac59b575b/extract/0.log"
Jul 15 16:12:22.136552 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.136481    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4_b46c748a-0c0e-45fb-8a85-0ba04cb78361/util/0.log"
Jul 15 16:12:22.144070 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.144042    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4_b46c748a-0c0e-45fb-8a85-0ba04cb78361/pull/0.log"
Jul 15 16:12:22.150747 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.150729    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4_b46c748a-0c0e-45fb-8a85-0ba04cb78361/extract/0.log"
Jul 15 16:12:22.262632 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.262605    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z_3fe1f4e4-3a2d-4173-9714-e239fb122f32/util/0.log"
Jul 15 16:12:22.268900 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.268881    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z_3fe1f4e4-3a2d-4173-9714-e239fb122f32/pull/0.log"
Jul 15 16:12:22.275260 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.275243    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z_3fe1f4e4-3a2d-4173-9714-e239fb122f32/extract/0.log"
Jul 15 16:12:22.648043 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:22.648014    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dz7wx_b755299d-7f20-496c-bd1b-93a92d33c8b5/manager/0.log"
Jul 15 16:12:23.038711 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:23.038634    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l6bh9_fce97a94-e8cf-4726-990a-75adc00740e6/manager/0.log"
Jul 15 16:12:23.150524 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:23.150494    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6hw2j_3e35a104-6234-4a2b-9402-3472949cda4e/limitador/0.log"
Jul 15 16:12:25.174979 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:25.174949    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwrlmr_d922bffe-518d-4aad-abf3-635a0eadfb32/main/0.log"
Jul 15 16:12:25.411675 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:25.411646    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-ptbsc_b51ddc1c-b7bc-4b8e-b049-42e6fc18aad8/main/0.log"
Jul 15 16:12:30.792300 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.792216    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xf74v/must-gather-pr7xm"]
Jul 15 16:12:30.792862 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.792842    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc8e64f6-5e68-40fe-9dea-49abc8da23c5" containerName="revoke-keys"
Jul 15 16:12:30.792967 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.792865    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc8e64f6-5e68-40fe-9dea-49abc8da23c5" containerName="revoke-keys"
Jul 15 16:12:30.792967 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.792893    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ebe502-ab66-4481-b216-7a405e57ba0c" containerName="maas-api"
Jul 15 16:12:30.792967 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.792902    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ebe502-ab66-4481-b216-7a405e57ba0c" containerName="maas-api"
Jul 15 16:12:30.793133 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.793054    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc8e64f6-5e68-40fe-9dea-49abc8da23c5" containerName="revoke-keys"
Jul 15 16:12:30.793133 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.793074    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ebe502-ab66-4481-b216-7a405e57ba0c" containerName="maas-api"
Jul 15 16:12:30.797032 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.797008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:30.799717 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.799694    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xf74v\"/\"openshift-service-ca.crt\""
Jul 15 16:12:30.799830 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.799723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xf74v\"/\"default-dockercfg-76hpf\""
Jul 15 16:12:30.799830 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.799744    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xf74v\"/\"kube-root-ca.crt\""
Jul 15 16:12:30.806634 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.806611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xf74v/must-gather-pr7xm"]
Jul 15 16:12:30.891114 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.891075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9077942d-3408-490f-aafb-74278c371b63-must-gather-output\") pod \"must-gather-pr7xm\" (UID: \"9077942d-3408-490f-aafb-74278c371b63\") " pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:30.891294 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.891174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dwlk\" (UniqueName: \"kubernetes.io/projected/9077942d-3408-490f-aafb-74278c371b63-kube-api-access-7dwlk\") pod \"must-gather-pr7xm\" (UID: \"9077942d-3408-490f-aafb-74278c371b63\") " pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:30.992431 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.992390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9077942d-3408-490f-aafb-74278c371b63-must-gather-output\") pod \"must-gather-pr7xm\" (UID: \"9077942d-3408-490f-aafb-74278c371b63\") " pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:30.992603 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.992460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwlk\" (UniqueName: \"kubernetes.io/projected/9077942d-3408-490f-aafb-74278c371b63-kube-api-access-7dwlk\") pod \"must-gather-pr7xm\" (UID: \"9077942d-3408-490f-aafb-74278c371b63\") " pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:30.992795 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:30.992773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9077942d-3408-490f-aafb-74278c371b63-must-gather-output\") pod \"must-gather-pr7xm\" (UID: \"9077942d-3408-490f-aafb-74278c371b63\") " pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:31.001123 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:31.001104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dwlk\" (UniqueName: \"kubernetes.io/projected/9077942d-3408-490f-aafb-74278c371b63-kube-api-access-7dwlk\") pod \"must-gather-pr7xm\" (UID: \"9077942d-3408-490f-aafb-74278c371b63\") " pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:31.107604 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:31.107573    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xf74v/must-gather-pr7xm"
Jul 15 16:12:31.250029 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:31.250001    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xf74v/must-gather-pr7xm"]
Jul 15 16:12:31.251299 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:12:31.251273    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9077942d_3408_490f_aafb_74278c371b63.slice/crio-2afa1294ca8136dac7a18ea074188db647f639df6729366f8c5df91a401348ca WatchSource:0}: Error finding container 2afa1294ca8136dac7a18ea074188db647f639df6729366f8c5df91a401348ca: Status 404 returned error can't find the container with id 2afa1294ca8136dac7a18ea074188db647f639df6729366f8c5df91a401348ca
Jul 15 16:12:32.217603 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:32.217572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xf74v/must-gather-pr7xm" event={"ID":"9077942d-3408-490f-aafb-74278c371b63","Type":"ContainerStarted","Data":"2afa1294ca8136dac7a18ea074188db647f639df6729366f8c5df91a401348ca"}
Jul 15 16:12:33.230536 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:33.230475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xf74v/must-gather-pr7xm" event={"ID":"9077942d-3408-490f-aafb-74278c371b63","Type":"ContainerStarted","Data":"f89d2052400f7728f46cb9aee2e6196d915f3d86a4c070725d6aaeb90deb0bf6"}
Jul 15 16:12:33.231034 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:33.230554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xf74v/must-gather-pr7xm" event={"ID":"9077942d-3408-490f-aafb-74278c371b63","Type":"ContainerStarted","Data":"4e6ce9b2199808bc28c0172b81b9d6bc9086411a08ace40ef1edd42dd2af61e6"}
Jul 15 16:12:33.248790 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:33.248708    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xf74v/must-gather-pr7xm" podStartSLOduration=2.346067834 podStartE2EDuration="3.248690748s" totalImagesPullingTime="902.622914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:12:33.248082839 +0000 UTC m=+2878.049542655" watchObservedRunningTime="2026-07-15 16:12:33.248690748 +0000 UTC m=+2878.050150563"
Jul 15 16:12:33.776253 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:33.776224    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vlhsk_2ca195cd-f793-4606-b4d9-a8048bc5e102/global-pull-secret-syncer/0.log"
Jul 15 16:12:33.870345 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:33.870315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mwv45_30334585-bcd0-44b9-bdf0-0677a905c124/konnectivity-agent/0.log"
Jul 15 16:12:33.929267 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:33.929240    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-125.ec2.internal_7b07821de4720a9cf3f16450e3c5031e/haproxy/0.log"
Jul 15 16:12:37.860880 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:37.860829    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v_a42fccbc-1d50-4ae0-91a2-763b44861dd9/extract/0.log"
Jul 15 16:12:37.885430 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:37.885403    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v_a42fccbc-1d50-4ae0-91a2-763b44861dd9/util/0.log"
Jul 15 16:12:37.908945 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:37.908906    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqz6v_a42fccbc-1d50-4ae0-91a2-763b44861dd9/pull/0.log"
Jul 15 16:12:37.943269 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:37.943212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc_d158794c-7d7d-42c8-8364-72aac59b575b/extract/0.log"
Jul 15 16:12:37.968123 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:37.968072    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc_d158794c-7d7d-42c8-8364-72aac59b575b/util/0.log"
Jul 15 16:12:37.991188 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:37.991154    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mlmdc_d158794c-7d7d-42c8-8364-72aac59b575b/pull/0.log"
Jul 15 16:12:38.019761 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.019721    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4_b46c748a-0c0e-45fb-8a85-0ba04cb78361/extract/0.log"
Jul 15 16:12:38.044262 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.044234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4_b46c748a-0c0e-45fb-8a85-0ba04cb78361/util/0.log"
Jul 15 16:12:38.067456 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.067392    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v66b4_b46c748a-0c0e-45fb-8a85-0ba04cb78361/pull/0.log"
Jul 15 16:12:38.094622 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.094592    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z_3fe1f4e4-3a2d-4173-9714-e239fb122f32/extract/0.log"
Jul 15 16:12:38.121164 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.121085    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z_3fe1f4e4-3a2d-4173-9714-e239fb122f32/util/0.log"
Jul 15 16:12:38.145936 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.145899    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14pf5z_3fe1f4e4-3a2d-4173-9714-e239fb122f32/pull/0.log"
Jul 15 16:12:38.678853 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.678823    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dz7wx_b755299d-7f20-496c-bd1b-93a92d33c8b5/manager/0.log"
Jul 15 16:12:38.914774 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.914470    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l6bh9_fce97a94-e8cf-4726-990a-75adc00740e6/manager/0.log"
Jul 15 16:12:38.936935 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:38.936847    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6hw2j_3e35a104-6234-4a2b-9402-3472949cda4e/limitador/0.log"
Jul 15 16:12:40.523394 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.523334    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc19423b-4abd-4cb3-a8b9-126734fdadaa/alertmanager/0.log"
Jul 15 16:12:40.547347 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.547319    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc19423b-4abd-4cb3-a8b9-126734fdadaa/config-reloader/0.log"
Jul 15 16:12:40.572652 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.572625    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc19423b-4abd-4cb3-a8b9-126734fdadaa/kube-rbac-proxy-web/0.log"
Jul 15 16:12:40.595553 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.595528    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc19423b-4abd-4cb3-a8b9-126734fdadaa/kube-rbac-proxy/0.log"
Jul 15 16:12:40.620742 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.620716    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc19423b-4abd-4cb3-a8b9-126734fdadaa/kube-rbac-proxy-metric/0.log"
Jul 15 16:12:40.645125 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.645096    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc19423b-4abd-4cb3-a8b9-126734fdadaa/prom-label-proxy/0.log"
Jul 15 16:12:40.671274 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.671249    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc19423b-4abd-4cb3-a8b9-126734fdadaa/init-config-reloader/0.log"
Jul 15 16:12:40.740154 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.740101    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-qnbsh_f1dcacea-7f36-47c3-9e55-935a6320fc76/kube-state-metrics/0.log"
Jul 15 16:12:40.760179 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.760126    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-qnbsh_f1dcacea-7f36-47c3-9e55-935a6320fc76/kube-rbac-proxy-main/0.log"
Jul 15 16:12:40.790161 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.790079    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-qnbsh_f1dcacea-7f36-47c3-9e55-935a6320fc76/kube-rbac-proxy-self/0.log"
Jul 15 16:12:40.819589 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.819543    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-774f6f965c-45gpn_a961e48d-8f70-4e29-9b66-b997e3f96c7c/metrics-server/0.log"
Jul 15 16:12:40.852656 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:40.852612    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-25kfx_ffdff59c-f9ad-4301-bfae-21a665db1481/monitoring-plugin/0.log"
Jul 15 16:12:41.105212 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.105174    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r78gp_67c16a39-6b90-49af-8513-497d3f8e49b7/node-exporter/0.log"
Jul 15 16:12:41.127924 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.127868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r78gp_67c16a39-6b90-49af-8513-497d3f8e49b7/kube-rbac-proxy/0.log"
Jul 15 16:12:41.151138 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.151106    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r78gp_67c16a39-6b90-49af-8513-497d3f8e49b7/init-textfile/0.log"
Jul 15 16:12:41.185394 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.185355    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wvvjn_ed72d9d4-8ef9-4b37-986c-b66253087631/kube-rbac-proxy-main/0.log"
Jul 15 16:12:41.210742 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.210707    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wvvjn_ed72d9d4-8ef9-4b37-986c-b66253087631/kube-rbac-proxy-self/0.log"
Jul 15 16:12:41.233201 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.233167    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wvvjn_ed72d9d4-8ef9-4b37-986c-b66253087631/openshift-state-metrics/0.log"
Jul 15 16:12:41.446564 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.446484    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-tpqjq_b3031f1f-e0ba-4095-a829-1d1c512ba618/prometheus-operator/0.log"
Jul 15 16:12:41.467226 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.467194    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-tpqjq_b3031f1f-e0ba-4095-a829-1d1c512ba618/kube-rbac-proxy/0.log"
Jul 15 16:12:41.505049 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.505015    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-7lg2h_0977e6dc-43b0-4918-a5af-6c47be1cf669/prometheus-operator-admission-webhook/0.log"
Jul 15 16:12:41.538782 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.538726    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cb9b7db46-lmzq7_e677916f-866a-4fc4-9f68-06c938d16d30/telemeter-client/0.log"
Jul 15 16:12:41.582511 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.582482    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cb9b7db46-lmzq7_e677916f-866a-4fc4-9f68-06c938d16d30/reload/0.log"
Jul 15 16:12:41.608170 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.608138    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cb9b7db46-lmzq7_e677916f-866a-4fc4-9f68-06c938d16d30/kube-rbac-proxy/0.log"
Jul 15 16:12:41.641110 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.641077    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-864774d8b5-ztztl_e3ee76aa-7994-48ee-b164-186cdf3ecff1/thanos-query/0.log"
Jul 15 16:12:41.668486 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.668447    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-864774d8b5-ztztl_e3ee76aa-7994-48ee-b164-186cdf3ecff1/kube-rbac-proxy-web/0.log"
Jul 15 16:12:41.696940 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.696860    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-864774d8b5-ztztl_e3ee76aa-7994-48ee-b164-186cdf3ecff1/kube-rbac-proxy/0.log"
Jul 15 16:12:41.721713 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.721680    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-864774d8b5-ztztl_e3ee76aa-7994-48ee-b164-186cdf3ecff1/prom-label-proxy/0.log"
Jul 15 16:12:41.745442 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.745404    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-864774d8b5-ztztl_e3ee76aa-7994-48ee-b164-186cdf3ecff1/kube-rbac-proxy-rules/0.log"
Jul 15 16:12:41.769739 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:41.769709    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-864774d8b5-ztztl_e3ee76aa-7994-48ee-b164-186cdf3ecff1/kube-rbac-proxy-metrics/0.log"
Jul 15 16:12:43.655528 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.655488    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"]
Jul 15 16:12:43.661392 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.661363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.670262 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.670234    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"]
Jul 15 16:12:43.745138 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.745103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sz5d\" (UniqueName: \"kubernetes.io/projected/05e70016-0f78-41aa-bacd-da24f6f7d2cd-kube-api-access-6sz5d\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.745342 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.745267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-lib-modules\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.745342 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.745315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-proc\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.745476 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.745346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-sys\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.745476 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.745401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-podres\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846237 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-podres\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846437 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sz5d\" (UniqueName: \"kubernetes.io/projected/05e70016-0f78-41aa-bacd-da24f6f7d2cd-kube-api-access-6sz5d\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846437 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-lib-modules\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846437 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-podres\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846437 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-proc\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846437 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-sys\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846703 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846524    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-sys\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846703 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-lib-modules\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.846703 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.846676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05e70016-0f78-41aa-bacd-da24f6f7d2cd-proc\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.855958 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.855934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sz5d\" (UniqueName: \"kubernetes.io/projected/05e70016-0f78-41aa-bacd-da24f6f7d2cd-kube-api-access-6sz5d\") pod \"perf-node-gather-daemonset-vttc7\" (UID: \"05e70016-0f78-41aa-bacd-da24f6f7d2cd\") " pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:43.980473 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:43.980388    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:44.188693 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:44.184894    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"]
Jul 15 16:12:44.190351 ip-10-0-135-125 kubenswrapper[2580]: W0715 16:12:44.190321    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod05e70016_0f78_41aa_bacd_da24f6f7d2cd.slice/crio-916a726c68029b2c9fdf82317014d1888df7062952b2d73637007fae8e87199a WatchSource:0}: Error finding container 916a726c68029b2c9fdf82317014d1888df7062952b2d73637007fae8e87199a: Status 404 returned error can't find the container with id 916a726c68029b2c9fdf82317014d1888df7062952b2d73637007fae8e87199a
Jul 15 16:12:44.278168 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:44.278138    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c9c55965-gwdg8_678240ac-3a0f-4287-92d3-fa1b5b6a79bc/console/0.log"
Jul 15 16:12:44.352881 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:44.352846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7" event={"ID":"05e70016-0f78-41aa-bacd-da24f6f7d2cd","Type":"ContainerStarted","Data":"fbbcdff8784c5cd378a42cc51f29e8c85d0568b0f148d121f844128838f34d9d"}
Jul 15 16:12:44.354715 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:44.354686    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7" event={"ID":"05e70016-0f78-41aa-bacd-da24f6f7d2cd","Type":"ContainerStarted","Data":"916a726c68029b2c9fdf82317014d1888df7062952b2d73637007fae8e87199a"}
Jul 15 16:12:44.354955 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:44.354933    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:44.376669 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:44.376610    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7" podStartSLOduration=1.3765928139999999 podStartE2EDuration="1.376592814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:12:44.376397637 +0000 UTC m=+2889.177857464" watchObservedRunningTime="2026-07-15 16:12:44.376592814 +0000 UTC m=+2889.178052629"
Jul 15 16:12:45.997610 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:45.997580    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rk8x5_47d26eda-8254-44a1-853c-5a5cd65b16e0/dns/0.log"
Jul 15 16:12:46.020114 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:46.020065    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rk8x5_47d26eda-8254-44a1-853c-5a5cd65b16e0/kube-rbac-proxy/0.log"
Jul 15 16:12:46.117013 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:46.116988    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x5qct_6cdc1e12-fcd9-4084-903f-e4ea77df2fab/dns-node-resolver/0.log"
Jul 15 16:12:46.664300 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:46.664252    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5d866f7d78-dnd5t_f0ed422c-9464-4d54-8062-4f53fcce1489/registry/0.log"
Jul 15 16:12:46.711948 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:46.711920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g2qfq_6c7aa5bc-94bc-44db-956a-c7b126adcdf3/node-ca/0.log"
Jul 15 16:12:48.758844 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:48.758810    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrkcw_06e689da-a2a5-4e5c-84e4-51b9327d778c/serve-healthcheck-canary/0.log"
Jul 15 16:12:49.270997 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:49.270971    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5pmqj_e83d2d43-3886-433f-a47c-cef5d46c90f7/kube-rbac-proxy/0.log"
Jul 15 16:12:49.293578 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:49.293552    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5pmqj_e83d2d43-3886-433f-a47c-cef5d46c90f7/exporter/0.log"
Jul 15 16:12:49.316921 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:49.316894    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5pmqj_e83d2d43-3886-433f-a47c-cef5d46c90f7/extractor/0.log"
Jul 15 16:12:51.378578 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:51.378543    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xf74v/perf-node-gather-daemonset-vttc7"
Jul 15 16:12:51.741019 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:51.740869    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ktlm2_b8330683-b064-4759-8568-4ebf931d0c21/manager/0.log"
Jul 15 16:12:52.198645 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:52.198619    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-997df6bcb-twxtg_f42580a0-134b-4913-9b8a-61f3b6a61485/manager/0.log"
Jul 15 16:12:52.216822 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:52.216797    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-997df6bcb-twxtg_f42580a0-134b-4913-9b8a-61f3b6a61485/copy-manifests/0.log"
Jul 15 16:12:53.656435 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:53.656405    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c5d886c95-4b4gn_12dc964a-65ac-4793-bde3-52733e840edb/manager/0.log"
Jul 15 16:12:53.709600 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:12:53.709572    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xm4l4_044c6944-9a79-4c7b-8b44-0261ae4f50e4/openshift-lws-operator/0.log"
Jul 15 16:13:00.372193 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.372165    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-47z5k_baeece6e-85ae-48ff-8df3-50336c6cc138/kube-multus-additional-cni-plugins/0.log"
Jul 15 16:13:00.395852 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.395825    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-47z5k_baeece6e-85ae-48ff-8df3-50336c6cc138/egress-router-binary-copy/0.log"
Jul 15 16:13:00.418539 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.418513    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-47z5k_baeece6e-85ae-48ff-8df3-50336c6cc138/cni-plugins/0.log"
Jul 15 16:13:00.440290 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.440261    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-47z5k_baeece6e-85ae-48ff-8df3-50336c6cc138/bond-cni-plugin/0.log"
Jul 15 16:13:00.463056 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.463036    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-47z5k_baeece6e-85ae-48ff-8df3-50336c6cc138/routeoverride-cni/0.log"
Jul 15 16:13:00.485345 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.485322    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-47z5k_baeece6e-85ae-48ff-8df3-50336c6cc138/whereabouts-cni-bincopy/0.log"
Jul 15 16:13:00.508618 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.508598    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-47z5k_baeece6e-85ae-48ff-8df3-50336c6cc138/whereabouts-cni/0.log"
Jul 15 16:13:00.859634 ip-10-0-135-125 kubenswrapper[2580]: I0715 16:13:00.859607    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f9gnk_1a3149f3-f5d7-40f0-921f-6f78f88b88ff/kube-multus/0.log"