Jul 09 19:39:48.489939 ip-10-0-136-128 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 19:39:48.844056 ip-10-0-136-128 kubenswrapper[2527]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 19:39:48.844056 ip-10-0-136-128 kubenswrapper[2527]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 19:39:48.844056 ip-10-0-136-128 kubenswrapper[2527]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 19:39:48.844056 ip-10-0-136-128 kubenswrapper[2527]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 19:39:48.844056 ip-10-0-136-128 kubenswrapper[2527]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 19:39:48.845040 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.844949    2527 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 19:39:48.850000 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.849977    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 19:39:48.850000 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.849998    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 19:39:48.850000 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850006    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850009    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850013    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850016    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850019    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850022    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850025    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850028    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850030    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850033    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850036    2527 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850038    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850041    2527 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850046    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850048    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850051    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850054    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850057    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850060    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850063    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 19:39:48.850154 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850066    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850068    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850071    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850074    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850076    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850081    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850084    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850087    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850089    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850092    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850094    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850097    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850099    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850103    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850106    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850109    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850113    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850116    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850121    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850123    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 19:39:48.850618 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850126    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850129    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850132    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850134    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850137    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850141    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850144    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850148    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850150    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850154    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850159    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850162    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850168    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850171    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850174    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850177    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850180    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850184    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850186    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850189    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 19:39:48.851147 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850192    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850195    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850200    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850203    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850205    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850208    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850212    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850216    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850218    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850221    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850223    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850226    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850229    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850232    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850237    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850239    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850242    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850246    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850249    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 19:39:48.851612 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850251    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850254    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850256    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850259    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850261    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850264    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850267    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850272    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850275    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850278    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850281    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850283    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850286    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850290    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850293    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850296    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850299    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.850301    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851474    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 19:39:48.852080 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851485    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851489    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851492    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851495    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851498    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851501    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851504    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851507    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851509    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851512    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851514    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851517    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851519    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851522    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851524    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851527    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851529    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851532    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851535    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851537    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 19:39:48.852531 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851539    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851542    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851544    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851547    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851550    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851553    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851555    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851558    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851560    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851562    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851566    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851570    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851574    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851576    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851579    2527 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851581    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851584    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851587    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851589    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851591    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 19:39:48.853051 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851594    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851596    2527 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851599    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851601    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851604    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851606    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851609    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851612    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851614    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851618    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851620    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851623    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851626    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851628    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851631    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851633    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851635    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851638    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851640    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 19:39:48.853542 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851658    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851661    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851665    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851668    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851671    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851675    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851678    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851681    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851684    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851687    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851689    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851692    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851695    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851698    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851701    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851704    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851706    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851709    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851712    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851715    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 19:39:48.854048 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851717    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851720    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851723    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851727    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851731    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851734    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851737    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851739    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851742    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851744    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851747    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851750    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851752    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851755    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851757    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851760    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851762    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851766    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.851769    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852150    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 19:39:48.854525 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852155    2527 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852157    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852160    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852163    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852166    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852169    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852171    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852173    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852176    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852178    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852181    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852183    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852186    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852188    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852191    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852194    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852196    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852199    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852201    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852204    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 19:39:48.855060 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852206    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852208    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852211    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852213    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852216    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852218    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852220    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852223    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852225    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852228    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852231    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852233    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852238    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852241    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852244    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852246    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852250    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852252    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852255    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852257    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 19:39:48.855533 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852260    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852262    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852265    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852267    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852269    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852272    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852274    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852276    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852279    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852281    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852284    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852286    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852289    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852291    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852293    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852296    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852298    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852301    2527 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852303    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852306    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 19:39:48.856037 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852308    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852311    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852313    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852316    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852318    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852320    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852323    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852325    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852327    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852330    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852332    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852335    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852337    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852340    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852342    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852344    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852347    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852351    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852353    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 19:39:48.856554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852356    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852359    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852363    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852366    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852368    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852372    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852375    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852378    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852380    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852383    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852385    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852387    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852390    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852392    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852395    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852398    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852400    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852403    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.852405    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 19:39:48.857171 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852474    2527 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852481    2527 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852488    2527 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852492    2527 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852496    2527 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852499    2527 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852503    2527 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852507    2527 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852510    2527 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852513    2527 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852517    2527 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852520    2527 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852523    2527 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852526    2527 flags.go:64] FLAG: --cgroup-root=""
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852528    2527 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852531    2527 flags.go:64] FLAG: --client-ca-file=""
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852535    2527 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852538    2527 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852542    2527 flags.go:64] FLAG: --cluster-domain=""
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852545    2527 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852548    2527 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852551    2527 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852554    2527 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852562    2527 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 19:39:48.857618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852565    2527 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852568    2527 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852571    2527 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852577    2527 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852581    2527 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852584    2527 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852587    2527 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852589    2527 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852593    2527 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852596    2527 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852599    2527 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852601    2527 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852604    2527 flags.go:64] FLAG: --enable-server="true"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852607    2527 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852611    2527 flags.go:64] FLAG: --event-burst="100"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852615    2527 flags.go:64] FLAG: --event-qps="50"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852617    2527 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852620    2527 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852623    2527 flags.go:64] FLAG: --eviction-hard=""
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852626    2527 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852629    2527 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852632    2527 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852634    2527 flags.go:64] FLAG: --eviction-soft=""
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852637    2527 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852640    2527 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 19:39:48.858230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852662    2527 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852665    2527 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852667    2527 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852670    2527 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852673    2527 flags.go:64] FLAG: --feature-gates=""
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852677    2527 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852687    2527 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852692    2527 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852695    2527 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852698    2527 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852702    2527 flags.go:64] FLAG: --help="false"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852705    2527 flags.go:64] FLAG: --hostname-override="ip-10-0-136-128.ec2.internal"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852708    2527 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852711    2527 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852714    2527 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852717    2527 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852721    2527 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852723    2527 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852726    2527 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852729    2527 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852731    2527 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852734    2527 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852737    2527 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852740    2527 flags.go:64] FLAG: --kube-reserved=""
Jul 09 19:39:48.858862 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852742    2527 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852745    2527 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852748    2527 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852750    2527 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852753    2527 flags.go:64] FLAG: --lock-file=""
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852755    2527 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852758    2527 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852761    2527 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852766    2527 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852769    2527 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852772    2527 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852774    2527 flags.go:64] FLAG: --logging-format="text"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852778    2527 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852782    2527 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852784    2527 flags.go:64] FLAG: --manifest-url=""
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852787    2527 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852791    2527 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852794    2527 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852799    2527 flags.go:64] FLAG: --max-pods="110"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852802    2527 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852805    2527 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852808    2527 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852810    2527 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852813    2527 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852832    2527 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 19:39:48.859485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852836    2527 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852844    2527 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852847    2527 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852850    2527 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852854    2527 flags.go:64] FLAG: --pod-cidr=""
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852857    2527 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852862    2527 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852865    2527 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852868    2527 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852871    2527 flags.go:64] FLAG: --port="10250"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852874    2527 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852877    2527 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-092406430504cef78"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852880    2527 flags.go:64] FLAG: --qos-reserved=""
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852883    2527 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852885    2527 flags.go:64] FLAG: --register-node="true"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852888    2527 flags.go:64] FLAG: --register-with-taints=""
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852891    2527 flags.go:64] FLAG: --registry-burst="10"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852894    2527 flags.go:64] FLAG: --registry-qps="5"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852897    2527 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852900    2527 flags.go:64] FLAG: --reserved-memory=""
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852904    2527 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852907    2527 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852910    2527 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852913    2527 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852916    2527 flags.go:64] FLAG: --runonce="false"
Jul 09 19:39:48.860111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852920    2527 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852923    2527 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852926    2527 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852928    2527 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852931    2527 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852934    2527 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852937    2527 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852939    2527 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852942    2527 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852945    2527 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852947    2527 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852950    2527 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852953    2527 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852956    2527 flags.go:64] FLAG: --system-cgroups=""
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852958    2527 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852963    2527 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852966    2527 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852969    2527 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852972    2527 flags.go:64] FLAG: --tls-min-version=""
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852975    2527 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852977    2527 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852980    2527 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852983    2527 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852985    2527 flags.go:64] FLAG: --v="2"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852989    2527 flags.go:64] FLAG: --version="false"
Jul 09 19:39:48.860756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852993    2527 flags.go:64] FLAG: --vmodule=""
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.852997    2527 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.853001    2527 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853115    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853119    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853122    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853125    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853128    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853131    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853134    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853136    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853141    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853145    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853147    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853150    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853152    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853154    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853157    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853159    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853162    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 19:39:48.861358 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853165    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853167    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853169    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853172    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853174    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853177    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853179    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853182    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853184    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853187    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853189    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853192    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853194    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853197    2527 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853201    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853204    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853206    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853210    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853214    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 19:39:48.862006 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853218    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853221    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853224    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853227    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853230    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853232    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853235    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853237    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853240    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853242    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853244    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853247    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853249    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853252    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853254    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853257    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853259    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853261    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853264    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 19:39:48.862458 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853266    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853269    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853271    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853273    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853276    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853278    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853281    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853283    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853289    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853292    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853294    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853297    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853299    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853302    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853304    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853306    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853309    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853312    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853314    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853316    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 19:39:48.862938 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853319    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853321    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853324    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853326    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853328    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853331    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853334    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853336    2527 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853338    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853341    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853344    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853347    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853350    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853353    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853355    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853358    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853361    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853364    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853367    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853369    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 19:39:48.863421 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853373    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853376    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853379    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.853381    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.853957    2527 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.861074    2527 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.861091    2527 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.861113    2527 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.861120    2527 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861176    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861182    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861185    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861189    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861191    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861194    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 19:39:48.863913 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861197    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861200    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861203    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861206    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861209    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861212    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861214    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861217    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861219    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861222    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861225    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861227    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861230    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861232    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861235    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861237    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861240    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861242    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861244    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 19:39:48.864292 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861247    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861249    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861252    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861254    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861258    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861261    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861264    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861266    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861270    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861274    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861277    2527 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861280    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861282    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861286    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861289    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861292    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861295    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861298    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861300    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861303    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 19:39:48.864805 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861305    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861308    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861312    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861315    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861318    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861320    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861323    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861326    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861328    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861330    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861333    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861335    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861338    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861340    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861343    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861345    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861348    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861352    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861354    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 19:39:48.865288 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861357    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861359    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861362    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861364    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861367    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861370    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861372    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861375    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861378    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861380    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861383    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861385    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861388    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861390    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861393    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861395    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861397    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861400    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861402    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 19:39:48.865827 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861405    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861407    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861410    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861412    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861414    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861417    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861419    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861422    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861424    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861426    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861430    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861432    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861435    2527 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861438    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861441    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 19:39:48.866325 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861443    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.861448    2527 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861557    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861562    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861565    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861568    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861571    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861573    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861576    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861578    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861581    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861583    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861585    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861589    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 19:39:48.866758 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861593    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861596    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861598    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861601    2527 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861604    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861606    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861609    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861611    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861615    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861617    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861620    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861622    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861624    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861627    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861630    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861633    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861636    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861638    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861658    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861662    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 19:39:48.867204 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861664    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861667    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861669    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861672    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861674    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861677    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861679    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861681    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861684    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861686    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861689    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861691    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861694    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861696    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861698    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861701    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861703    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861706    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861708    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861711    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 19:39:48.867797 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861713    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861716    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861718    2527 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861720    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861723    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861725    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861728    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861731    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861733    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861737    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861739    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861742    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861744    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861746    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861750    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861752    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861754    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861757    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861759    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861762    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 19:39:48.868580 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861764    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861767    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861770    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861773    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861776    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861778    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861780    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861783    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861785    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861789    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861793    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861795    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861798    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861800    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861803    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861806    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861808    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861811    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861814    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 19:39:48.869152 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861816    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861818    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861821    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861824    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861826    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861829    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861831    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:48.861834    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.861839    2527 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.861973    2527 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.864155    2527 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.865619    2527 server.go:1019] "Starting client certificate rotation"
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.865753    2527 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 19:39:48.869837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.865809    2527 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 19:39:48.870209 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.868324    2527 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 19:39:48.871575 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.871560    2527 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 19:39:48.881012 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.880992    2527 log.go:25] "Validated CRI v1 runtime API"
Jul 09 19:39:48.884115 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.884093    2527 log.go:25] "Validated CRI v1 image API"
Jul 09 19:39:48.884972 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.884955    2527 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 19:39:48.887222 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.887203    2527 fs.go:135] Filesystem UUIDs: map[041447b4-8c90-4830-866e-f61a5ce9eaea:/dev/nvme0n1p3 71809914-bbb5-43dd-813f-8fc934d1e927:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 09 19:39:48.887268 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.887223    2527 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 19:39:48.892316 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.892208    2527 manager.go:216] Machine: {Timestamp:2026-07-09 19:39:48.891050278 +0000 UTC m=+0.292185086 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099719 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d2558b66be5c90fdcfb8cd9ec2dd6 SystemUUID:ec2d2558-b66b-e5c9-0fdc-fb8cd9ec2dd6 BootID:367911bd-29fa-4016-a807-b1f7c8e8ca90 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c4:fe:15:9f:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c4:fe:15:9f:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:1c:ee:23:25:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 19:39:48.892316 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.892315    2527 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 19:39:48.892430 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.892396    2527 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 19:39:48.895138 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.895114    2527 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 19:39:48.895279 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.895141    2527 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-128.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 19:39:48.895812 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.895802    2527 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 19:39:48.895846 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.895814    2527 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 19:39:48.895846 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.895821    2527 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 19:39:48.895961 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.895945    2527 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 19:39:48.896438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.896429    2527 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 19:39:48.897383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.897372    2527 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 19:39:48.901393 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.901381    2527 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 19:39:48.901713 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.901703    2527 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 19:39:48.901749 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.901735    2527 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 19:39:48.901749 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.901746    2527 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 19:39:48.901803 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.901760    2527 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 19:39:48.901803 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.901770    2527 kubelet.go:398] "Adding apiserver pod source"
Jul 09 19:39:48.901803 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.901779    2527 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 19:39:48.902136 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.902123    2527 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 19:39:48.902195 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.902142    2527 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 19:39:48.904148 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.904130    2527 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 19:39:48.905780 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.905764    2527 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 19:39:48.905833 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.905809    2527 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 19:39:48.906418 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906404    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906421    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906427    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906433    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906442    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906447    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906456    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906461    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 19:39:48.906464 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906467    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 19:39:48.906700 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906472    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 19:39:48.906700 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906480    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 19:39:48.906700 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.906485    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 19:39:48.907097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.907087    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 19:39:48.907128 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.907098    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 19:39:48.910539 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.910525    2527 server.go:1268] "Started kubelet"
Jul 09 19:39:48.910706 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.910633    2527 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 19:39:48.910755 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.910677    2527 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 19:39:48.910755 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.910737    2527 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 19:39:48.911507 ip-10-0-136-128 systemd[1]: Started Kubernetes Kubelet.
Jul 09 19:39:48.912411 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.912396    2527 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 19:39:48.912509 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.912475    2527 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-128.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 19:39:48.912509 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.912468    2527 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 19:39:48.912600 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.912573    2527 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 19:39:48.915990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.915973    2527 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 19:39:48.916405 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.916385    2527 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 19:39:48.919009 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.918991    2527 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 19:39:48.919098 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919045    2527 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 19:39:48.919156 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.919104    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:48.919680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919660    2527 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 19:39:48.919680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919681    2527 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 19:39:48.919825 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919762    2527 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 19:39:48.919825 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919772    2527 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 19:39:48.919825 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919763    2527 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 19:39:48.919951 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919858    2527 factory.go:55] Registering systemd factory
Jul 09 19:39:48.919951 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.919870    2527 factory.go:223] Registration of the systemd container factory successfully
Jul 09 19:39:48.920107 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.920091    2527 factory.go:153] Registering CRI-O factory
Jul 09 19:39:48.920176 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.920109    2527 factory.go:223] Registration of the crio container factory successfully
Jul 09 19:39:48.920176 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.920163    2527 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 19:39:48.920243 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.920186    2527 factory.go:103] Registering Raw factory
Jul 09 19:39:48.920243 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.920201    2527 manager.go:1196] Started watching for new ooms in manager
Jul 09 19:39:48.920327 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.919132    2527 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-128.ec2.internal.18c0b663092a5733  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-128.ec2.internal,UID:ip-10-0-136-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-128.ec2.internal,},FirstTimestamp:2026-07-09 19:39:48.910503731 +0000 UTC m=+0.311638531,LastTimestamp:2026-07-09 19:39:48.910503731 +0000 UTC m=+0.311638531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-128.ec2.internal,}"
Jul 09 19:39:48.920570 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.920559    2527 manager.go:349] Starting recovery of all containers
Jul 09 19:39:48.927000 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.926811    2527 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 19:39:48.927293 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.927268    2527 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 19:39:48.929825 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.929790    2527 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sh9kq"
Jul 09 19:39:48.931225 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.931207    2527 manager.go:354] Recovery completed
Jul 09 19:39:48.932969 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.932947    2527 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 09 19:39:48.935906 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.935894    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 19:39:48.937037 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.937022    2527 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sh9kq"
Jul 09 19:39:48.938333 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.938319    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 19:39:48.938376 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.938347    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 19:39:48.938376 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.938359    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientPID"
Jul 09 19:39:48.938921 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.938908    2527 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 19:39:48.938921 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.938917    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 19:39:48.939011 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.938920    2527 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 19:39:48.939011 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.938952    2527 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 19:39:48.939615 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.939601    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 19:39:48.939722 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.939625    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 19:39:48.939722 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.939635    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientPID"
Jul 09 19:39:48.940138 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.940063    2527 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-128.ec2.internal.18c0b6630ad2f8ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-128.ec2.internal,UID:ip-10-0-136-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-128.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-128.ec2.internal,},FirstTimestamp:2026-07-09 19:39:48.938332366 +0000 UTC m=+0.339467170,LastTimestamp:2026-07-09 19:39:48.938332366 +0000 UTC m=+0.339467170,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-128.ec2.internal,}"
Jul 09 19:39:48.940996 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.940984    2527 policy_none.go:49] "None policy: Start"
Jul 09 19:39:48.941033 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.941002    2527 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 19:39:48.941033 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.941011    2527 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 19:39:48.943039 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.943025    2527 policy_none.go:47] "Start"
Jul 09 19:39:48.984254 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.984236    2527 manager.go:337] "Starting Device Plugin manager"
Jul 09 19:39:48.984377 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.984271    2527 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 19:39:48.984377 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.984281    2527 server.go:87] "Starting device plugin registration server"
Jul 09 19:39:48.984563 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.984550    2527 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 19:39:48.984597 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.984568    2527 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 19:39:48.984709 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.984691    2527 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 19:39:48.984790 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.984776    2527 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 19:39:48.984790 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:48.984785    2527 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 19:39:48.985396 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.985378    2527 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 19:39:48.985473 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:48.985419    2527 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.069274 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.069234    2527 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 19:39:49.071517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.070621    2527 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 19:39:49.071517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.070681    2527 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 19:39:49.071517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.070708    2527 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 19:39:49.071517 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.070751    2527 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 19:39:49.073277 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.073254    2527 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 19:39:49.084755 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.084738    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 19:39:49.085779 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.085762    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 19:39:49.085850 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.085795    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 19:39:49.085850 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.085805    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientPID"
Jul 09 19:39:49.085850 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.085826    2527 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.094208 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.094159    2527 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.094208 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.094180    2527 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-128.ec2.internal\": node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.111275 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.111252    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.170904 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.170869    2527 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal"]
Jul 09 19:39:49.170997 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.170948    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 19:39:49.172565 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.172548    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 19:39:49.172657 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.172577    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 19:39:49.172657 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.172586    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientPID"
Jul 09 19:39:49.174973 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.174961    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 19:39:49.175109 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175078    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.175146 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175121    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 19:39:49.175729 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175714    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 19:39:49.175800 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175742    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 19:39:49.175800 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175748    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 19:39:49.175800 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175756    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientPID"
Jul 09 19:39:49.175800 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175766    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 19:39:49.175800 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.175775    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientPID"
Jul 09 19:39:49.177958 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.177941    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.178025 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.177973    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 19:39:49.178682 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.178666    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 19:39:49.178760 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.178695    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 19:39:49.178760 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.178709    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeHasSufficientPID"
Jul 09 19:39:49.197459 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.197432    2527 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-128.ec2.internal\" not found" node="ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.201810 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.201790    2527 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-128.ec2.internal\" not found" node="ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.211861 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.211838    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.221151 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.221128    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aafd699d8d5c0e0776d82dc30235bc75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal\" (UID: \"aafd699d8d5c0e0776d82dc30235bc75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.221214 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.221160    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aafd699d8d5c0e0776d82dc30235bc75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal\" (UID: \"aafd699d8d5c0e0776d82dc30235bc75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.221214 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.221184    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98ef2af5604f9c52421ea4f57e9babc4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.ec2.internal\" (UID: \"98ef2af5604f9c52421ea4f57e9babc4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.312491 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.312461    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.321420 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.321397    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aafd699d8d5c0e0776d82dc30235bc75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal\" (UID: \"aafd699d8d5c0e0776d82dc30235bc75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.321472 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.321456    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aafd699d8d5c0e0776d82dc30235bc75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal\" (UID: \"aafd699d8d5c0e0776d82dc30235bc75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.321505 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.321468    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aafd699d8d5c0e0776d82dc30235bc75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal\" (UID: \"aafd699d8d5c0e0776d82dc30235bc75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.321552 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.321485    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98ef2af5604f9c52421ea4f57e9babc4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.ec2.internal\" (UID: \"98ef2af5604f9c52421ea4f57e9babc4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.321552 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.321534    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aafd699d8d5c0e0776d82dc30235bc75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal\" (UID: \"aafd699d8d5c0e0776d82dc30235bc75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.321615 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.321564    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98ef2af5604f9c52421ea4f57e9babc4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.ec2.internal\" (UID: \"98ef2af5604f9c52421ea4f57e9babc4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.412893 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.412831    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.501368 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.501329    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.504495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.504470    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:49.513262 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.513242    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.613877 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.613836    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.714388 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.714313    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.814928 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.814895    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.865389 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.865356    2527 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 19:39:49.865956 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.865496    2527 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 19:39:49.915570 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:49.915538    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-128.ec2.internal\" not found"
Jul 09 19:39:49.942778 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.942743    2527 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 19:34:49 +0000 UTC" deadline="2027-12-10 17:36:38.807437497 +0000 UTC"
Jul 09 19:39:49.942778 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.942771    2527 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12453h56m48.864669233s"
Jul 09 19:39:49.981691 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:49.981597    2527 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 19:39:50.019605 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.019577    2527 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:50.030594 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.030566    2527 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 19:39:50.032187 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.032171    2527 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal"
Jul 09 19:39:50.048148 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.048124    2527 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 19:39:50.095032 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:50.094984    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ef2af5604f9c52421ea4f57e9babc4.slice/crio-f7102f9c076a029a621874d42a1c8902f7c26a65f5aee17c8378fa0a2e4d4e67 WatchSource:0}: Error finding container f7102f9c076a029a621874d42a1c8902f7c26a65f5aee17c8378fa0a2e4d4e67: Status 404 returned error can't find the container with id f7102f9c076a029a621874d42a1c8902f7c26a65f5aee17c8378fa0a2e4d4e67
Jul 09 19:39:50.095252 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:50.095232    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafd699d8d5c0e0776d82dc30235bc75.slice/crio-f8e2b853dd321baa3e8a4ec10608e26b9e109abda941a0bb60cd0e9cc2e7dfde WatchSource:0}: Error finding container f8e2b853dd321baa3e8a4ec10608e26b9e109abda941a0bb60cd0e9cc2e7dfde: Status 404 returned error can't find the container with id f8e2b853dd321baa3e8a4ec10608e26b9e109abda941a0bb60cd0e9cc2e7dfde
Jul 09 19:39:50.100647 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.100628    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 19:39:50.289485 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.289390    2527 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 19:39:50.467999 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.467970    2527 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 19:39:50.821326 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.821288    2527 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 19:39:50.902753 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.902720    2527 apiserver.go:52] "Watching apiserver"
Jul 09 19:39:50.911999 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.911968    2527 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 19:39:50.912447 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.912407    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bjpjv","kube-system/konnectivity-agent-s4l69","openshift-cluster-node-tuning-operator/tuned-cxgwl","openshift-image-registry/node-ca-nfv4x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal","openshift-multus/multus-6t2f8","openshift-network-diagnostics/network-check-target-zxkds","openshift-ovn-kubernetes/ovnkube-node-nlhxk","kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6","openshift-dns/node-resolver-sd999","openshift-multus/multus-additional-cni-plugins-9c2zf","openshift-multus/network-metrics-daemon-42kwj","openshift-network-operator/iptables-alerter-mg2tt"]
Jul 09 19:39:50.915265 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.915228    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:50.915393 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:50.915336    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:39:50.917555 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.917532    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:50.921024 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.921002    2527 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 19:39:50.921127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.921041    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 19:39:50.921188 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.921133    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.921188 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.921148    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r5h4k\""
Jul 09 19:39:50.921293 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.921263    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 19:39:50.922198 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.922175    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:50.923843 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.923826    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p8vtr\""
Jul 09 19:39:50.923938 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.923858    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 19:39:50.923938 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.923878    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 19:39:50.924818 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.924471    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:50.924818 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.924668    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 19:39:50.925194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.925174    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 19:39:50.925332 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.925311    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 19:39:50.925440 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.925424    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4gd8m\""
Jul 09 19:39:50.927002 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.926981    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:50.928874 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.928853    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 19:39:50.929291 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.929272    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-szg8g\""
Jul 09 19:39:50.929501 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.929485    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 19:39:50.929691 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.929676    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 19:39:50.929920 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.929899    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:50.930920 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.930897    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:50.931013 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.930935    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9dadcbf5-686f-4448-ab73-d05ec262206a-hosts-file\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:50.931013 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.930971    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-kubernetes\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931013 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.930998    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysctl-conf\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931023    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-var-lib-kubelet\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931046    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea4bda2e-4a85-4410-88a0-843cae158512-host\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931071    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-sys\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931125    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c07c072-febf-4034-885d-5bdc60a49fe0-tmp\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931161    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dadcbf5-686f-4448-ab73-d05ec262206a-tmp-dir\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931190    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-systemd\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931194    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 19:39:50.931215 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931216    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-host\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931240    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-tuned\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931256    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931266    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm6rw\" (UniqueName: \"kubernetes.io/projected/3c07c072-febf-4034-885d-5bdc60a49fe0-kube-api-access-lm6rw\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931295    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wwkp\" (UniqueName: \"kubernetes.io/projected/ea4bda2e-4a85-4410-88a0-843cae158512-kube-api-access-9wwkp\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931320    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tm4s\" (UniqueName: \"kubernetes.io/projected/9dadcbf5-686f-4448-ab73-d05ec262206a-kube-api-access-4tm4s\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931346    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-modprobe-d\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931372    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysconfig\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931409    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysctl-d\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931436    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931458    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931436    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-run\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931572    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-lib-modules\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:50.931737 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931599    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ea4bda2e-4a85-4410-88a0-843cae158512-serviceca\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:50.932438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.931751    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sst4k\""
Jul 09 19:39:50.932438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.932040    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:50.932438 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:50.932105    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:39:50.933320 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.933298    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 19:39:50.933426 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.933328    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rddvz\""
Jul 09 19:39:50.933592 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.933571    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 19:39:50.933858 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.933839    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 19:39:50.933942 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.933869    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 19:39:50.934263 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.934109    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 19:39:50.934445 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.934427    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 19:39:50.943775 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.942524    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:50.943775 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.942594    2527 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 19:39:50.943775 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.943197    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:50.943775 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:50.943310    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:39:50.943775 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.943327    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:50.943775 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.943449    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:50.945644 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.945623    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 19:39:50.946077 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.946054    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 19:39:50.946173 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.946059    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-587lf\""
Jul 09 19:39:50.946245 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.946065    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 19:39:50.946910 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.946889    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5xzt2\""
Jul 09 19:39:50.947355 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.947181    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 19:39:50.947355 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.947233    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 19:39:50.947355 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.947260    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 19:39:50.947560 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.947413    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 19:39:50.947842 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.947821    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pc7kv\""
Jul 09 19:39:50.966569 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.965782    2527 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-klgzr"
Jul 09 19:39:50.972494 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:50.972465    2527 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-klgzr"
Jul 09 19:39:51.021527 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.021505    2527 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 19:39:51.032219 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032190    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0da2917-eb23-47fc-9db2-dd7737386e59-dbus\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.032362 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032233    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.032362 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032265    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm6rw\" (UniqueName: \"kubernetes.io/projected/3c07c072-febf-4034-885d-5bdc60a49fe0-kube-api-access-lm6rw\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.032362 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032291    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-cni-bin\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.032362 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032315    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-hostroot\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.032362 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032349    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djwfq\" (UniqueName: \"kubernetes.io/projected/b76d8e50-2d66-4da1-b546-db2837e4aaf9-kube-api-access-djwfq\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032374    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovnkube-config\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032397    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032422    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-cnibin\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032448    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-cni-binary-copy\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032475    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysconfig\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032501    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-lib-modules\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032529    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032571    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysctl-conf\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.032626 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032612    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd4sj\" (UniqueName: \"kubernetes.io/projected/b7a72406-3b3d-4535-b5ba-8376627edc03-kube-api-access-rd4sj\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032638    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/151b891f-3f38-4b7b-bad7-ceeb49ac642f-host-slash\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032664    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-env-overrides\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032698    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-netns\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032742    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-tuned\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032766    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-node-log\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032801    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-modprobe-d\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032827    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysctl-d\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032853    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ea4bda2e-4a85-4410-88a0-843cae158512-serviceca\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032888    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2rs6\" (UniqueName: \"kubernetes.io/projected/151b891f-3f38-4b7b-bad7-ceeb49ac642f-kube-api-access-m2rs6\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032914    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-systemd-units\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032942    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-cni-netd\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032970    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.032996    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9dadcbf5-686f-4448-ab73-d05ec262206a-hosts-file\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033021    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-var-lib-kubelet\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033046    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87f73e8f-a2a4-4dfd-a685-ba51aa038002-konnectivity-ca\") pod \"konnectivity-agent-s4l69\" (UID: \"87f73e8f-a2a4-4dfd-a685-ba51aa038002\") " pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:51.033097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033076    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033101    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033141    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-cni-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033169    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-kubelet\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033197    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-var-lib-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033224    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-system-cni-dir\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033252    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-systemd\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033277    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-host\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033303    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wwkp\" (UniqueName: \"kubernetes.io/projected/ea4bda2e-4a85-4410-88a0-843cae158512-kube-api-access-9wwkp\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033328    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-cnibin\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033354    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-cni-multus\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033384    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-conf-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033410    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87f73e8f-a2a4-4dfd-a685-ba51aa038002-agent-certs\") pod \"konnectivity-agent-s4l69\" (UID: \"87f73e8f-a2a4-4dfd-a685-ba51aa038002\") " pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033440    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tm4s\" (UniqueName: \"kubernetes.io/projected/9dadcbf5-686f-4448-ab73-d05ec262206a-kube-api-access-4tm4s\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033465    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-run\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033489    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-system-cni-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033527    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-ovn\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033554    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovn-node-metrics-cert\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033579    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovnkube-script-lib\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033604    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033627    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea4bda2e-4a85-4410-88a0-843cae158512-host\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033661    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-socket-dir-parent\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033687    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-k8s-cni-cncf-io\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033742    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-etc-kubernetes\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033771    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033797    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0da2917-eb23-47fc-9db2-dd7737386e59-kubelet-config\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033824    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-slash\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033856    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-run-netns\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033884    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-sys\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033910    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-os-release\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033937    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-daemon-config\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033964    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-multus-certs\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.033991    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-systemd\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034024    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-log-socket\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.034900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034049    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-cni-bin\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034077    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf7qv\" (UniqueName: \"kubernetes.io/projected/cc22fcf6-f091-4486-afcb-caf63a0ead86-kube-api-access-kf7qv\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034106    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dadcbf5-686f-4448-ab73-d05ec262206a-tmp-dir\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034134    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b76d8e50-2d66-4da1-b546-db2837e4aaf9-cni-binary-copy\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034167    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-os-release\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034209    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034238    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/151b891f-3f38-4b7b-bad7-ceeb49ac642f-iptables-alerter-script\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034267    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-etc-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034293    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-registration-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034329    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-device-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034357    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-sys-fs\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034385    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-kubernetes\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034424    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-kubelet\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034453    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-socket-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034480    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c07c072-febf-4034-885d-5bdc60a49fe0-tmp\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034513    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvdwk\" (UniqueName: \"kubernetes.io/projected/31b31734-c0f5-4653-9893-c3bb20fecb2c-kube-api-access-wvdwk\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:51.035708 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034542    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034571    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.034600    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drmnb\" (UniqueName: \"kubernetes.io/projected/3d3a1880-0c85-4596-b979-75786eec18a5-kube-api-access-drmnb\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035095    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysconfig\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035186    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea4bda2e-4a85-4410-88a0-843cae158512-host\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035232    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-lib-modules\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035307    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-sys\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035390    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysctl-conf\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035749    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9dadcbf5-686f-4448-ab73-d05ec262206a-tmp-dir\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035798    2527 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.035923    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-kubernetes\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036389    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-modprobe-d\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036383    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-systemd\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036456    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-var-lib-kubelet\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.036515 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036509    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-host\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.037231 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036541    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-sysctl-d\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.037231 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036739    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c07c072-febf-4034-885d-5bdc60a49fe0-run\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.037231 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036853    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9dadcbf5-686f-4448-ab73-d05ec262206a-hosts-file\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:51.037231 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.036957    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ea4bda2e-4a85-4410-88a0-843cae158512-serviceca\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:51.039599 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.039574    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c07c072-febf-4034-885d-5bdc60a49fe0-tmp\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.039599 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.039590    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c07c072-febf-4034-885d-5bdc60a49fe0-etc-tuned\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.043026 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.043004    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 19:39:51.043147 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.043033    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 19:39:51.043147 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.043046    2527 projected.go:196] Error preparing data for projected volume kube-api-access-rtgsh for pod openshift-network-diagnostics/network-check-target-zxkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:51.043147 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.043113    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh podName:389b1ece-cc85-4eb3-9111-a57d697c98c2 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:51.543092819 +0000 UTC m=+2.944227636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rtgsh" (UniqueName: "kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh") pod "network-check-target-zxkds" (UID: "389b1ece-cc85-4eb3-9111-a57d697c98c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:51.044967 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.044906    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm6rw\" (UniqueName: \"kubernetes.io/projected/3c07c072-febf-4034-885d-5bdc60a49fe0-kube-api-access-lm6rw\") pod \"tuned-cxgwl\" (UID: \"3c07c072-febf-4034-885d-5bdc60a49fe0\") " pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.046582 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.046560    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tm4s\" (UniqueName: \"kubernetes.io/projected/9dadcbf5-686f-4448-ab73-d05ec262206a-kube-api-access-4tm4s\") pod \"node-resolver-sd999\" (UID: \"9dadcbf5-686f-4448-ab73-d05ec262206a\") " pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:51.047372 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.047347    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wwkp\" (UniqueName: \"kubernetes.io/projected/ea4bda2e-4a85-4410-88a0-843cae158512-kube-api-access-9wwkp\") pod \"node-ca-nfv4x\" (UID: \"ea4bda2e-4a85-4410-88a0-843cae158512\") " pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:51.075643 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.075549    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal" event={"ID":"aafd699d8d5c0e0776d82dc30235bc75","Type":"ContainerStarted","Data":"f8e2b853dd321baa3e8a4ec10608e26b9e109abda941a0bb60cd0e9cc2e7dfde"}
Jul 09 19:39:51.076755 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.076729    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal" event={"ID":"98ef2af5604f9c52421ea4f57e9babc4","Type":"ContainerStarted","Data":"f7102f9c076a029a621874d42a1c8902f7c26a65f5aee17c8378fa0a2e4d4e67"}
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.134913    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.134965    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/151b891f-3f38-4b7b-bad7-ceeb49ac642f-iptables-alerter-script\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.134994    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-etc-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135019    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-registration-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135045    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-device-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135074    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-sys-fs\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135104    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-kubelet\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135130    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-socket-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135156    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvdwk\" (UniqueName: \"kubernetes.io/projected/31b31734-c0f5-4653-9893-c3bb20fecb2c-kube-api-access-wvdwk\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135181    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135208    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135234    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drmnb\" (UniqueName: \"kubernetes.io/projected/3d3a1880-0c85-4596-b979-75786eec18a5-kube-api-access-drmnb\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135260    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0da2917-eb23-47fc-9db2-dd7737386e59-dbus\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135289    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135315    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-cni-bin\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135339    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-hostroot\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.136271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135373    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djwfq\" (UniqueName: \"kubernetes.io/projected/b76d8e50-2d66-4da1-b546-db2837e4aaf9-kube-api-access-djwfq\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135391    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovnkube-config\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135406    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135420    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-cnibin\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135436    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-cni-binary-copy\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135468    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135497    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd4sj\" (UniqueName: \"kubernetes.io/projected/b7a72406-3b3d-4535-b5ba-8376627edc03-kube-api-access-rd4sj\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135522    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/151b891f-3f38-4b7b-bad7-ceeb49ac642f-host-slash\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135546    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-env-overrides\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135563    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-netns\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135580    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-node-log\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135603    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rs6\" (UniqueName: \"kubernetes.io/projected/151b891f-3f38-4b7b-bad7-ceeb49ac642f-kube-api-access-m2rs6\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135619    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-systemd-units\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135636    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-cni-netd\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135679    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87f73e8f-a2a4-4dfd-a685-ba51aa038002-konnectivity-ca\") pod \"konnectivity-agent-s4l69\" (UID: \"87f73e8f-a2a4-4dfd-a685-ba51aa038002\") " pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135706    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135746    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.137306 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135763    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-cni-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135778    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-kubelet\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135806    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-var-lib-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135822    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-system-cni-dir\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135842    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-cnibin\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135858    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-cni-multus\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135886    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-conf-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135912    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87f73e8f-a2a4-4dfd-a685-ba51aa038002-agent-certs\") pod \"konnectivity-agent-s4l69\" (UID: \"87f73e8f-a2a4-4dfd-a685-ba51aa038002\") " pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135940    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-system-cni-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135964    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-ovn\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135985    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovn-node-metrics-cert\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136002    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovnkube-script-lib\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136019    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136038    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-socket-dir-parent\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136058    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-k8s-cni-cncf-io\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136086    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-etc-kubernetes\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136114    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:51.138194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136119    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136141    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0da2917-eb23-47fc-9db2-dd7737386e59-kubelet-config\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136162    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-slash\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136190    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-run-netns\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136209    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-os-release\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136231    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-daemon-config\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136258    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-multus-certs\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136289    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-systemd\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136315    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-log-socket\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136347    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-cni-bin\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136365    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf7qv\" (UniqueName: \"kubernetes.io/projected/cc22fcf6-f091-4486-afcb-caf63a0ead86-kube-api-access-kf7qv\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136390    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b76d8e50-2d66-4da1-b546-db2837e4aaf9-cni-binary-copy\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136418    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-os-release\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136460    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-etc-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136553    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-os-release\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.136891    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0da2917-eb23-47fc-9db2-dd7737386e59-dbus\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.136991    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:51.139094 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.137070    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret podName:d0da2917-eb23-47fc-9db2-dd7737386e59 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:51.637051624 +0000 UTC m=+3.038186417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret") pod "global-pull-secret-syncer-bjpjv" (UID: "d0da2917-eb23-47fc-9db2-dd7737386e59") : object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137104    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137166    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-registration-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137213    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-device-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137284    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-hostroot\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137330    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-cni-bin\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137379    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-cni-multus\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137404    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-socket-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137471    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-conf-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137636    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-sys-fs\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137694    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-var-lib-kubelet\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137748    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-systemd-units\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137783    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137810    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-cnibin\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137911    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.135612    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137950    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovnkube-config\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.139985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.137979    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-cni-netd\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138021    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-cni-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138096    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-kubelet\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138141    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-var-lib-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138185    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7a72406-3b3d-4535-b5ba-8376627edc03-system-cni-dir\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138235    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-cnibin\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138282    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-slash\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138334    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-system-cni-dir\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138380    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-ovn\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138425    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7a72406-3b3d-4535-b5ba-8376627edc03-cni-binary-copy\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138485    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87f73e8f-a2a4-4dfd-a685-ba51aa038002-konnectivity-ca\") pod \"konnectivity-agent-s4l69\" (UID: \"87f73e8f-a2a4-4dfd-a685-ba51aa038002\") " pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138536    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138539    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-openvswitch\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138592    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-netns\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138633    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-node-log\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.138956    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-env-overrides\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139026    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/151b891f-3f38-4b7b-bad7-ceeb49ac642f-host-slash\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139087    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-run-systemd\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.140631 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139140    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d3a1880-0c85-4596-b979-75786eec18a5-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139198    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-socket-dir-parent\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139242    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/151b891f-3f38-4b7b-bad7-ceeb49ac642f-iptables-alerter-script\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139435    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-log-socket\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139485    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-cni-bin\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139632    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovnkube-script-lib\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139697    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-multus-certs\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139754    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-host-run-k8s-cni-cncf-io\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139820    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b76d8e50-2d66-4da1-b546-db2837e4aaf9-multus-daemon-config\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.139834    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.139893    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs podName:31b31734-c0f5-4653-9893-c3bb20fecb2c nodeName:}" failed. No retries permitted until 2026-07-09 19:39:51.63987674 +0000 UTC m=+3.041011531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs") pod "network-metrics-daemon-42kwj" (UID: "31b31734-c0f5-4653-9893-c3bb20fecb2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139891    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc22fcf6-f091-4486-afcb-caf63a0ead86-host-run-netns\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139934    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-etc-kubernetes\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139950    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b76d8e50-2d66-4da1-b546-db2837e4aaf9-cni-binary-copy\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139960    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b76d8e50-2d66-4da1-b546-db2837e4aaf9-os-release\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.139989    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0da2917-eb23-47fc-9db2-dd7737386e59-kubelet-config\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.141508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.141080    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87f73e8f-a2a4-4dfd-a685-ba51aa038002-agent-certs\") pod \"konnectivity-agent-s4l69\" (UID: \"87f73e8f-a2a4-4dfd-a685-ba51aa038002\") " pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:51.142335 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.142319    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc22fcf6-f091-4486-afcb-caf63a0ead86-ovn-node-metrics-cert\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.146272 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.146217    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drmnb\" (UniqueName: \"kubernetes.io/projected/3d3a1880-0c85-4596-b979-75786eec18a5-kube-api-access-drmnb\") pod \"aws-ebs-csi-driver-node-ckbj6\" (UID: \"3d3a1880-0c85-4596-b979-75786eec18a5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.147528 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.147499    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djwfq\" (UniqueName: \"kubernetes.io/projected/b76d8e50-2d66-4da1-b546-db2837e4aaf9-kube-api-access-djwfq\") pod \"multus-6t2f8\" (UID: \"b76d8e50-2d66-4da1-b546-db2837e4aaf9\") " pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.148035 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.148005    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd4sj\" (UniqueName: \"kubernetes.io/projected/b7a72406-3b3d-4535-b5ba-8376627edc03-kube-api-access-rd4sj\") pod \"multus-additional-cni-plugins-9c2zf\" (UID: \"b7a72406-3b3d-4535-b5ba-8376627edc03\") " pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.148143 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.148127    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2rs6\" (UniqueName: \"kubernetes.io/projected/151b891f-3f38-4b7b-bad7-ceeb49ac642f-kube-api-access-m2rs6\") pod \"iptables-alerter-mg2tt\" (UID: \"151b891f-3f38-4b7b-bad7-ceeb49ac642f\") " pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.149285 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.149262    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf7qv\" (UniqueName: \"kubernetes.io/projected/cc22fcf6-f091-4486-afcb-caf63a0ead86-kube-api-access-kf7qv\") pod \"ovnkube-node-nlhxk\" (UID: \"cc22fcf6-f091-4486-afcb-caf63a0ead86\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.149511 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.149495    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvdwk\" (UniqueName: \"kubernetes.io/projected/31b31734-c0f5-4653-9893-c3bb20fecb2c-kube-api-access-wvdwk\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:51.247321 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.247291    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sd999"
Jul 09 19:39:51.253877 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.253852    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cxgwl"
Jul 09 19:39:51.262874 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.262851    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nfv4x"
Jul 09 19:39:51.270904 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.270879    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mg2tt"
Jul 09 19:39:51.278908 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.278886    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6t2f8"
Jul 09 19:39:51.288904 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.288881    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:39:51.296740 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.296718    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:39:51.304497 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.304474    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6"
Jul 09 19:39:51.311535 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.311512    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9c2zf"
Jul 09 19:39:51.640375 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.640333    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:51.640584 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.640395    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:51.640584 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.640439    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:51.640584 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640527    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 19:39:51.640584 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640549    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:51.640584 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640559    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 19:39:51.640584 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640578    2527 projected.go:196] Error preparing data for projected volume kube-api-access-rtgsh for pod openshift-network-diagnostics/network-check-target-zxkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:51.640915 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640614    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret podName:d0da2917-eb23-47fc-9db2-dd7737386e59 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:52.640593186 +0000 UTC m=+4.041728001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret") pod "global-pull-secret-syncer-bjpjv" (UID: "d0da2917-eb23-47fc-9db2-dd7737386e59") : object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:51.640915 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640641    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh podName:389b1ece-cc85-4eb3-9111-a57d697c98c2 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:52.640619845 +0000 UTC m=+4.041754635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rtgsh" (UniqueName: "kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh") pod "network-check-target-zxkds" (UID: "389b1ece-cc85-4eb3-9111-a57d697c98c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:51.640915 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640739    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:51.640915 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:51.640775    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs podName:31b31734-c0f5-4653-9893-c3bb20fecb2c nodeName:}" failed. No retries permitted until 2026-07-09 19:39:52.640766335 +0000 UTC m=+4.041901124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs") pod "network-metrics-daemon-42kwj" (UID: "31b31734-c0f5-4653-9893-c3bb20fecb2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:51.977821 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.977781    2527 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 19:34:51 +0000 UTC" deadline="2027-12-07 13:58:57.497129347 +0000 UTC"
Jul 09 19:39:51.978304 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:51.977818    2527 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12378h19m5.519317059s"
Jul 09 19:39:52.072074 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:52.072037    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:52.072287 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.072195    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:39:52.346684 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:52.346427    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc22fcf6_f091_4486_afcb_caf63a0ead86.slice/crio-ea9ec647183c5f7d26c0ce0eb33e881ec84a3ba550f81b2f77466b5056199fe8 WatchSource:0}: Error finding container ea9ec647183c5f7d26c0ce0eb33e881ec84a3ba550f81b2f77466b5056199fe8: Status 404 returned error can't find the container with id ea9ec647183c5f7d26c0ce0eb33e881ec84a3ba550f81b2f77466b5056199fe8
Jul 09 19:39:52.348863 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:52.348837    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151b891f_3f38_4b7b_bad7_ceeb49ac642f.slice/crio-2295fe2089ae4028bc3ab98363120f8ea86899643d81a04c195d170af5000dd2 WatchSource:0}: Error finding container 2295fe2089ae4028bc3ab98363120f8ea86899643d81a04c195d170af5000dd2: Status 404 returned error can't find the container with id 2295fe2089ae4028bc3ab98363120f8ea86899643d81a04c195d170af5000dd2
Jul 09 19:39:52.352505 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:52.352483    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f73e8f_a2a4_4dfd_a685_ba51aa038002.slice/crio-ef5a8794717fad04efbdce4dec1fc199b8a56f8405ae9fdd599cfa574fa1687e WatchSource:0}: Error finding container ef5a8794717fad04efbdce4dec1fc199b8a56f8405ae9fdd599cfa574fa1687e: Status 404 returned error can't find the container with id ef5a8794717fad04efbdce4dec1fc199b8a56f8405ae9fdd599cfa574fa1687e
Jul 09 19:39:52.354254 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:52.354230    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a72406_3b3d_4535_b5ba_8376627edc03.slice/crio-595b9d4dbcf7c0a9d76038784e38c69a4291ac50bbc3f572adc710c531422d0f WatchSource:0}: Error finding container 595b9d4dbcf7c0a9d76038784e38c69a4291ac50bbc3f572adc710c531422d0f: Status 404 returned error can't find the container with id 595b9d4dbcf7c0a9d76038784e38c69a4291ac50bbc3f572adc710c531422d0f
Jul 09 19:39:52.354925 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:52.354819    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4bda2e_4a85_4410_88a0_843cae158512.slice/crio-425f4fb341e670c5af83a103901281be1360e4b377a55d01bf81fe5f19129d1f WatchSource:0}: Error finding container 425f4fb341e670c5af83a103901281be1360e4b377a55d01bf81fe5f19129d1f: Status 404 returned error can't find the container with id 425f4fb341e670c5af83a103901281be1360e4b377a55d01bf81fe5f19129d1f
Jul 09 19:39:52.356501 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:52.356463    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76d8e50_2d66_4da1_b546_db2837e4aaf9.slice/crio-286ff8a35ca2a6a55a249ad3cd7dc237a799da5fc67ed5e49ce45e70568041e2 WatchSource:0}: Error finding container 286ff8a35ca2a6a55a249ad3cd7dc237a799da5fc67ed5e49ce45e70568041e2: Status 404 returned error can't find the container with id 286ff8a35ca2a6a55a249ad3cd7dc237a799da5fc67ed5e49ce45e70568041e2
Jul 09 19:39:52.359024 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:39:52.359000    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3a1880_0c85_4596_b979_75786eec18a5.slice/crio-e12740eac5977b074c21d7210638a7efea207f197368ba7cff72fee6198ce3aa WatchSource:0}: Error finding container e12740eac5977b074c21d7210638a7efea207f197368ba7cff72fee6198ce3aa: Status 404 returned error can't find the container with id e12740eac5977b074c21d7210638a7efea207f197368ba7cff72fee6198ce3aa
Jul 09 19:39:52.649122 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:52.649080    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:52.649294 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:52.649153    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:52.649294 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:52.649194    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:52.649294 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649264    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:52.649478 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649297    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 19:39:52.649478 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649302    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:52.649478 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649313    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 19:39:52.649478 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649324    2527 projected.go:196] Error preparing data for projected volume kube-api-access-rtgsh for pod openshift-network-diagnostics/network-check-target-zxkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:52.649478 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649354    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs podName:31b31734-c0f5-4653-9893-c3bb20fecb2c nodeName:}" failed. No retries permitted until 2026-07-09 19:39:54.649329618 +0000 UTC m=+6.050464432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs") pod "network-metrics-daemon-42kwj" (UID: "31b31734-c0f5-4653-9893-c3bb20fecb2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:52.649478 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649376    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret podName:d0da2917-eb23-47fc-9db2-dd7737386e59 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:54.649365418 +0000 UTC m=+6.050500207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret") pod "global-pull-secret-syncer-bjpjv" (UID: "d0da2917-eb23-47fc-9db2-dd7737386e59") : object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:52.649478 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:52.649392    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh podName:389b1ece-cc85-4eb3-9111-a57d697c98c2 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:54.649383614 +0000 UTC m=+6.050518402 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rtgsh" (UniqueName: "kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh") pod "network-check-target-zxkds" (UID: "389b1ece-cc85-4eb3-9111-a57d697c98c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:52.978997 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:52.978874    2527 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 19:34:51 +0000 UTC" deadline="2027-12-22 12:21:40.240146242 +0000 UTC"
Jul 09 19:39:52.978997 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:52.978943    2527 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12736h41m47.261208116s"
Jul 09 19:39:53.073972 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.073934    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:53.074149 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:53.074081    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:39:53.074502 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.074481    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:53.074600 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:53.074585    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:39:53.093786 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.093745    2527 generic.go:356] "Generic (PLEG): container finished" podID="aafd699d8d5c0e0776d82dc30235bc75" containerID="7b5fbcae3e0f0dc7c0b17794715899857d6bba6b99fd87bb8db4d845145fe661" exitCode=0
Jul 09 19:39:53.093956 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.093840    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal" event={"ID":"aafd699d8d5c0e0776d82dc30235bc75","Type":"ContainerDied","Data":"7b5fbcae3e0f0dc7c0b17794715899857d6bba6b99fd87bb8db4d845145fe661"}
Jul 09 19:39:53.100673 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.100588    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cxgwl" event={"ID":"3c07c072-febf-4034-885d-5bdc60a49fe0","Type":"ContainerStarted","Data":"5c8378e9a69e1d994106f0dd968b9992c13d5a29700273b0365d7f830790ed28"}
Jul 09 19:39:53.104117 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.104025    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s4l69" event={"ID":"87f73e8f-a2a4-4dfd-a685-ba51aa038002","Type":"ContainerStarted","Data":"ef5a8794717fad04efbdce4dec1fc199b8a56f8405ae9fdd599cfa574fa1687e"}
Jul 09 19:39:53.107115 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.106592    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal" event={"ID":"98ef2af5604f9c52421ea4f57e9babc4","Type":"ContainerStarted","Data":"1969a4d18a13a6875a261283bab7eb1c259a6a1d75b93ff649750491c24072f0"}
Jul 09 19:39:53.110777 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.110069    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sd999" event={"ID":"9dadcbf5-686f-4448-ab73-d05ec262206a","Type":"ContainerStarted","Data":"5d09cc061fd0eb65a1b0f7686f3e6f815ff4f1494b5038bd22521abe6cfbf3fa"}
Jul 09 19:39:53.115046 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.115022    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6" event={"ID":"3d3a1880-0c85-4596-b979-75786eec18a5","Type":"ContainerStarted","Data":"e12740eac5977b074c21d7210638a7efea207f197368ba7cff72fee6198ce3aa"}
Jul 09 19:39:53.117468 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.117439    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6t2f8" event={"ID":"b76d8e50-2d66-4da1-b546-db2837e4aaf9","Type":"ContainerStarted","Data":"286ff8a35ca2a6a55a249ad3cd7dc237a799da5fc67ed5e49ce45e70568041e2"}
Jul 09 19:39:53.122104 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.122074    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nfv4x" event={"ID":"ea4bda2e-4a85-4410-88a0-843cae158512","Type":"ContainerStarted","Data":"425f4fb341e670c5af83a103901281be1360e4b377a55d01bf81fe5f19129d1f"}
Jul 09 19:39:53.125867 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.125815    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerStarted","Data":"595b9d4dbcf7c0a9d76038784e38c69a4291ac50bbc3f572adc710c531422d0f"}
Jul 09 19:39:53.130385 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.130355    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mg2tt" event={"ID":"151b891f-3f38-4b7b-bad7-ceeb49ac642f","Type":"ContainerStarted","Data":"2295fe2089ae4028bc3ab98363120f8ea86899643d81a04c195d170af5000dd2"}
Jul 09 19:39:53.132755 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:53.132654    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"ea9ec647183c5f7d26c0ce0eb33e881ec84a3ba550f81b2f77466b5056199fe8"}
Jul 09 19:39:54.071855 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:54.071817    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:54.072454 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.071973    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:39:54.153885 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:54.153792    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal" event={"ID":"aafd699d8d5c0e0776d82dc30235bc75","Type":"ContainerStarted","Data":"bda4ca97778644fbb5696c75195c79f45478c41b9b534383c2b33aa697d13815"}
Jul 09 19:39:54.193869 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:54.192883    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.ec2.internal" podStartSLOduration=4.192857822 podStartE2EDuration="4.192857822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:39:54.185442597 +0000 UTC m=+5.586577413" watchObservedRunningTime="2026-07-09 19:39:54.192857822 +0000 UTC m=+5.593992636"
Jul 09 19:39:54.193869 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:54.193364    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.ec2.internal" podStartSLOduration=4.193352471 podStartE2EDuration="4.193352471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:39:53.125783815 +0000 UTC m=+4.526918632" watchObservedRunningTime="2026-07-09 19:39:54.193352471 +0000 UTC m=+5.594487288"
Jul 09 19:39:54.672164 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:54.672112    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:54.672375 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:54.672193    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:54.672375 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:54.672242    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:54.672501 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.672402    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:54.672501 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.672468    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs podName:31b31734-c0f5-4653-9893-c3bb20fecb2c nodeName:}" failed. No retries permitted until 2026-07-09 19:39:58.672448918 +0000 UTC m=+10.073583714 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs") pod "network-metrics-daemon-42kwj" (UID: "31b31734-c0f5-4653-9893-c3bb20fecb2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:54.673363 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.672907    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:54.673363 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.672961    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret podName:d0da2917-eb23-47fc-9db2-dd7737386e59 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:58.672945659 +0000 UTC m=+10.074080453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret") pod "global-pull-secret-syncer-bjpjv" (UID: "d0da2917-eb23-47fc-9db2-dd7737386e59") : object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:54.673363 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.673042    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 19:39:54.673363 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.673056    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 19:39:54.673363 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.673070    2527 projected.go:196] Error preparing data for projected volume kube-api-access-rtgsh for pod openshift-network-diagnostics/network-check-target-zxkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:54.673363 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:54.673105    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh podName:389b1ece-cc85-4eb3-9111-a57d697c98c2 nodeName:}" failed. No retries permitted until 2026-07-09 19:39:58.67309321 +0000 UTC m=+10.074228003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rtgsh" (UniqueName: "kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh") pod "network-check-target-zxkds" (UID: "389b1ece-cc85-4eb3-9111-a57d697c98c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:55.074613 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:55.074511    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:55.075095 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:55.074661    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:39:55.075287 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:55.075239    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:55.075401 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:55.075356    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:39:56.071827 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:56.071793    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:56.071997 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:56.071956    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:39:57.072154 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:57.071680    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:57.072154 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:57.071813    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:39:57.072154 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:57.071876    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:57.072154 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:57.072030    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:39:58.071096 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:58.071054    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:58.071259 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.071201    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:39:58.706426 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:58.706384    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:58.706930 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:58.706453    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:39:58.706930 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:58.706523    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:58.706930 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.706731    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:58.706930 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.706797    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret podName:d0da2917-eb23-47fc-9db2-dd7737386e59 nodeName:}" failed. No retries permitted until 2026-07-09 19:40:06.706777237 +0000 UTC m=+18.107912028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret") pod "global-pull-secret-syncer-bjpjv" (UID: "d0da2917-eb23-47fc-9db2-dd7737386e59") : object "kube-system"/"original-pull-secret" not registered
Jul 09 19:39:58.707289 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.707271    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 19:39:58.707351 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.707297    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 19:39:58.707351 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.707310    2527 projected.go:196] Error preparing data for projected volume kube-api-access-rtgsh for pod openshift-network-diagnostics/network-check-target-zxkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:58.707443 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.707391    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh podName:389b1ece-cc85-4eb3-9111-a57d697c98c2 nodeName:}" failed. No retries permitted until 2026-07-09 19:40:06.707376022 +0000 UTC m=+18.108510809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rtgsh" (UniqueName: "kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh") pod "network-check-target-zxkds" (UID: "389b1ece-cc85-4eb3-9111-a57d697c98c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:39:58.707497 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.707460    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:58.707551 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:58.707498    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs podName:31b31734-c0f5-4653-9893-c3bb20fecb2c nodeName:}" failed. No retries permitted until 2026-07-09 19:40:06.707485003 +0000 UTC m=+18.108619790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs") pod "network-metrics-daemon-42kwj" (UID: "31b31734-c0f5-4653-9893-c3bb20fecb2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:39:59.073283 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:59.072730    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:39:59.073283 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:39:59.072750    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:39:59.073283 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:59.072891    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:39:59.073283 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:39:59.073032    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:00.071055 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:00.071021    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:00.071508 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:00.071160    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:01.074229 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:01.073764    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:01.074229 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:01.073772    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:01.074229 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:01.073897    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:01.074229 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:01.073990    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:02.071422 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:02.071379    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:02.071615 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:02.071526    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:03.071622 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.071364    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:03.072143 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:03.071806    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:03.072193 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.071551    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:03.072285 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:03.072256    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:03.175634 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.175593    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s4l69" event={"ID":"87f73e8f-a2a4-4dfd-a685-ba51aa038002","Type":"ContainerStarted","Data":"580b41436dc12f22c5a13a0646eccec9b850cca8d5275202633ab9d3c6eb4391"}
Jul 09 19:40:03.177340 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.177304    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sd999" event={"ID":"9dadcbf5-686f-4448-ab73-d05ec262206a","Type":"ContainerStarted","Data":"4a7a0dff014052be585a1307af96a8f76a7b57c9fc16a65866591a8d25bfe40d"}
Jul 09 19:40:03.180212 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.180185    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6" event={"ID":"3d3a1880-0c85-4596-b979-75786eec18a5","Type":"ContainerStarted","Data":"42ad3b071e0f3ef33513353ca86600a1eb2806ea425ea7defec3c73a341bc246"}
Jul 09 19:40:03.182178 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.182152    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nfv4x" event={"ID":"ea4bda2e-4a85-4410-88a0-843cae158512","Type":"ContainerStarted","Data":"22878e977f892512f260fe78a78fd38b4458af2f756b4c5c2f3de0c3eadf4720"}
Jul 09 19:40:03.183952 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.183928    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerStarted","Data":"078118ac4a5254d081e95c76ab0f24811eff94af1608742339448b8f194ebd41"}
Jul 09 19:40:03.185777 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.185732    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cxgwl" event={"ID":"3c07c072-febf-4034-885d-5bdc60a49fe0","Type":"ContainerStarted","Data":"58965cb3f6f5ce85d63817a81df136c0fe12062fc42083f51214d46c590e0f60"}
Jul 09 19:40:03.193613 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.193556    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s4l69" podStartSLOduration=3.819923541 podStartE2EDuration="14.193539555s" totalImagesPullingTime="10.373616014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:03.193146964 +0000 UTC m=+14.594281773" watchObservedRunningTime="2026-07-09 19:40:03.193539555 +0000 UTC m=+14.594674366"
Jul 09 19:40:03.229445 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.229391    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sd999" podStartSLOduration=3.837164054 podStartE2EDuration="14.229374826s" totalImagesPullingTime="10.392210772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:03.229045197 +0000 UTC m=+14.630180007" watchObservedRunningTime="2026-07-09 19:40:03.229374826 +0000 UTC m=+14.630509636"
Jul 09 19:40:03.248437 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.248335    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cxgwl" podStartSLOduration=3.832332831 podStartE2EDuration="14.248314621s" totalImagesPullingTime="10.41598179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:03.247248757 +0000 UTC m=+14.648383567" watchObservedRunningTime="2026-07-09 19:40:03.248314621 +0000 UTC m=+14.649449433"
Jul 09 19:40:03.263660 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:03.263589    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nfv4x" podStartSLOduration=3.897602133 podStartE2EDuration="14.263568749s" totalImagesPullingTime="10.365966616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:03.262791244 +0000 UTC m=+14.663926091" watchObservedRunningTime="2026-07-09 19:40:03.263568749 +0000 UTC m=+14.664703563"
Jul 09 19:40:04.071819 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:04.071787    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:04.072352 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:04.071919    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:05.071053 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:05.071018    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:05.071226 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:05.071019    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:05.071226 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:05.071156    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:05.071330 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:05.071250    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:05.192766 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:05.192725    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mg2tt" event={"ID":"151b891f-3f38-4b7b-bad7-ceeb49ac642f","Type":"ContainerStarted","Data":"c3d43e7a315cb7331695d54404d15e832d1dd624d8fdb8360902698066ff9e86"}
Jul 09 19:40:05.210151 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:05.210094    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mg2tt" podStartSLOduration=5.805754467 podStartE2EDuration="16.210076099s" totalImagesPullingTime="10.404321632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:05.209351225 +0000 UTC m=+16.610486036" watchObservedRunningTime="2026-07-09 19:40:05.210076099 +0000 UTC m=+16.611210959"
Jul 09 19:40:05.418618 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:05.418513    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:40:05.419453 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:05.419434    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:40:06.071462 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:06.071425    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:06.071636 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.071543    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:06.773226 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:06.773196    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:06.773246    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:06.773286    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773357    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773388    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773398    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773402    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773420    2527 projected.go:196] Error preparing data for projected volume kube-api-access-rtgsh for pod openshift-network-diagnostics/network-check-target-zxkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773430    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs podName:31b31734-c0f5-4653-9893-c3bb20fecb2c nodeName:}" failed. No retries permitted until 2026-07-09 19:40:22.773412303 +0000 UTC m=+34.174547105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs") pod "network-metrics-daemon-42kwj" (UID: "31b31734-c0f5-4653-9893-c3bb20fecb2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773447    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret podName:d0da2917-eb23-47fc-9db2-dd7737386e59 nodeName:}" failed. No retries permitted until 2026-07-09 19:40:22.773438733 +0000 UTC m=+34.174573522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret") pod "global-pull-secret-syncer-bjpjv" (UID: "d0da2917-eb23-47fc-9db2-dd7737386e59") : object "kube-system"/"original-pull-secret" not registered
Jul 09 19:40:06.773841 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:06.773465    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh podName:389b1ece-cc85-4eb3-9111-a57d697c98c2 nodeName:}" failed. No retries permitted until 2026-07-09 19:40:22.773451711 +0000 UTC m=+34.174586516 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rtgsh" (UniqueName: "kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh") pod "network-check-target-zxkds" (UID: "389b1ece-cc85-4eb3-9111-a57d697c98c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:40:07.074044 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:07.073349    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:07.074044 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:07.073472    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:07.074044 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:07.073852    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:07.074044 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:07.073939    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:08.071727 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:08.071681    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:08.072140 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:08.071817    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:09.072521 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:09.072307    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:09.072985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:09.072381    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:09.072985 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:09.072609    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:09.072985 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:09.072696    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:10.071899 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:10.071856    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:10.072095 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:10.072003    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:11.071405 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:11.071371    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:11.071906 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:11.071489    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:11.071906 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:11.071541    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:11.071906 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:11.071665    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:11.757850 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:11.757604    2527 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 19:40:11.995278 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:11.995167    2527 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T19:40:11.757828197Z","UUID":"82bfc350-d216-4388-9e32-03825ddc449d","Handler":null,"Name":"","Endpoint":""}
Jul 09 19:40:11.996742 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:11.996720    2527 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 19:40:11.996851 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:11.996766    2527 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 19:40:12.071930 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.071886    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:12.072352 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:12.072047    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:12.208843 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.208803    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6" event={"ID":"3d3a1880-0c85-4596-b979-75786eec18a5","Type":"ContainerStarted","Data":"0445f49e893ddad32ed8d55a9b6e53afdd157551e13f61d257e2c615ab5b49ca"}
Jul 09 19:40:12.210177 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.210114    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6t2f8" event={"ID":"b76d8e50-2d66-4da1-b546-db2837e4aaf9","Type":"ContainerStarted","Data":"909e83a2cba17454085fbc03c9b0efa998cd0248d413e998d4c245e96e5c92aa"}
Jul 09 19:40:12.211619 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.211592    2527 generic.go:356] "Generic (PLEG): container finished" podID="b7a72406-3b3d-4535-b5ba-8376627edc03" containerID="078118ac4a5254d081e95c76ab0f24811eff94af1608742339448b8f194ebd41" exitCode=0
Jul 09 19:40:12.211727 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.211668    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerDied","Data":"078118ac4a5254d081e95c76ab0f24811eff94af1608742339448b8f194ebd41"}
Jul 09 19:40:12.214619 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.214597    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"801205c5cf76231ac03c4af76b2e055cf3fe6af8e873a96d88b737fa6e44d837"}
Jul 09 19:40:12.214716 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.214693    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"72ff6c697447aa0f19f17b74a559aec687db9b1d9a3f11eaa8ba331c47e5c263"}
Jul 09 19:40:12.214777 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.214745    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"5bf6f5da9ce423b37d77d1a03c61154fc275e55fe581e186860ab81291450fbe"}
Jul 09 19:40:12.214831 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.214795    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"c24b2bb964e1c5abce0881d1e1f43bb825a436827f5468b372650a83f541d747"}
Jul 09 19:40:12.214884 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.214845    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"eddd866de0fb3355faa3d54b3ecad6344d73f0094b7a3e7b815d95d855a95e3d"}
Jul 09 19:40:12.214936 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.214896    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"cbb5fb34281d4e0372ed594f7a8199fb3387dc35f9de86dd3a8f49f6740618d9"}
Jul 09 19:40:12.229413 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:12.229360    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6t2f8" podStartSLOduration=3.989350332 podStartE2EDuration="23.22934759s" totalImagesPullingTime="19.239997258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:12.228451729 +0000 UTC m=+23.629586539" watchObservedRunningTime="2026-07-09 19:40:12.22934759 +0000 UTC m=+23.630482398"
Jul 09 19:40:13.071135 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:13.071106    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:13.071274 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:13.071215    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:13.071452 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:13.071432    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:13.071560 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:13.071542    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:13.218288 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:13.218254    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6" event={"ID":"3d3a1880-0c85-4596-b979-75786eec18a5","Type":"ContainerStarted","Data":"0f0021fe228fc11b014f891402f7c98722870efd67cd3c8dbb05fe98b8bdba63"}
Jul 09 19:40:13.243406 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:13.243351    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckbj6" podStartSLOduration=3.880537118 podStartE2EDuration="24.243330904s" totalImagesPullingTime="20.362793786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:13.243216046 +0000 UTC m=+24.644350854" watchObservedRunningTime="2026-07-09 19:40:13.243330904 +0000 UTC m=+24.644465698"
Jul 09 19:40:14.071547 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:14.071520    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:14.071704 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:14.071683    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:14.225260 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:14.225222    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"97a668c56d97db33e06ce61a66045bbf14e5e471e6a22ed983f82640cf8ff4a4"}
Jul 09 19:40:15.071714 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:15.071680    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:15.071714 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:15.071712    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:15.071917 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:15.071792    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:15.071917 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:15.071908    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:16.070977 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:16.070925    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:16.071542 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:16.071071    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:16.276782 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:16.276742    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:40:16.276971 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:16.276897    2527 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 09 19:40:16.277500 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:16.277476    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s4l69"
Jul 09 19:40:17.071140 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:17.071103    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:17.071616 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:17.071153    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:17.071616 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:17.071263    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:17.071616 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:17.071409    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:18.071771 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:18.071583    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:18.072116 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:18.071861    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:18.236110 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:18.236015    2527 generic.go:356] "Generic (PLEG): container finished" podID="b7a72406-3b3d-4535-b5ba-8376627edc03" containerID="9fc0e004697300f75525c5fd5dc82f1ebba35a63d6b8facad49a00f009dfd4c8" exitCode=0
Jul 09 19:40:18.236110 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:18.236090    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerDied","Data":"9fc0e004697300f75525c5fd5dc82f1ebba35a63d6b8facad49a00f009dfd4c8"}
Jul 09 19:40:18.239718 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:18.239698    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" event={"ID":"cc22fcf6-f091-4486-afcb-caf63a0ead86","Type":"ContainerStarted","Data":"59b5aac1cbd1fa098b62f3ec549844fd8f3747ddc3303b6c9197fdfa3b36ec8e"}
Jul 09 19:40:18.239978 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:18.239952    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:40:18.256864 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:18.256838    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:40:18.294222 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:18.294176    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk" podStartSLOduration=10.044100521 podStartE2EDuration="29.294162384s" totalImagesPullingTime="19.250061863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:18.293039378 +0000 UTC m=+29.694174187" watchObservedRunningTime="2026-07-09 19:40:18.294162384 +0000 UTC m=+29.695297194"
Jul 09 19:40:19.072512 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.072482    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:19.073021 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:19.072565    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:19.073021 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.072692    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:19.073021 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:19.072813    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:19.242998 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.242968    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:40:19.243140 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.243054    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:40:19.258428 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.258398    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:40:19.704679 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.704586    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zxkds"]
Jul 09 19:40:19.704796 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.704735    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:19.704840 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:19.704818    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:19.708100 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.708070    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bjpjv"]
Jul 09 19:40:19.708269 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.708183    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:19.708323 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:19.708293    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:19.708719 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.708701    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-42kwj"]
Jul 09 19:40:19.708804 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:19.708796    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:19.708898 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:19.708880    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:20.246581 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:20.246549    2527 generic.go:356] "Generic (PLEG): container finished" podID="b7a72406-3b3d-4535-b5ba-8376627edc03" containerID="54ba1e4b7a4693ccb7e35be6807e4504471940b8e5618ecbfac10aa246e7f9ff" exitCode=0
Jul 09 19:40:20.247039 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:20.246629    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerDied","Data":"54ba1e4b7a4693ccb7e35be6807e4504471940b8e5618ecbfac10aa246e7f9ff"}
Jul 09 19:40:21.071702 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:21.071676    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:21.071702 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:21.071701    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:21.071849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:21.071682    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:21.071849 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:21.071786    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:21.071956 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:21.071925    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:21.072079 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:21.072020    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:21.251030 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:21.250994    2527 generic.go:356] "Generic (PLEG): container finished" podID="b7a72406-3b3d-4535-b5ba-8376627edc03" containerID="87c34951b9267b28a70e09fa56f524df6b23409c35f2c871bd34555823daa082" exitCode=0
Jul 09 19:40:21.251487 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:21.251067    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerDied","Data":"87c34951b9267b28a70e09fa56f524df6b23409c35f2c871bd34555823daa082"}
Jul 09 19:40:22.790622 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:22.790580    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:22.790659    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:22.790703    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790799    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790831    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790845    2527 projected.go:196] Error preparing data for projected volume kube-api-access-rtgsh for pod openshift-network-diagnostics/network-check-target-zxkds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790855    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790807    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790904    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh podName:389b1ece-cc85-4eb3-9111-a57d697c98c2 nodeName:}" failed. No retries permitted until 2026-07-09 19:40:54.790883918 +0000 UTC m=+66.192018722 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rtgsh" (UniqueName: "kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh") pod "network-check-target-zxkds" (UID: "389b1ece-cc85-4eb3-9111-a57d697c98c2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790924    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs podName:31b31734-c0f5-4653-9893-c3bb20fecb2c nodeName:}" failed. No retries permitted until 2026-07-09 19:40:54.790913605 +0000 UTC m=+66.192048399 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs") pod "network-metrics-daemon-42kwj" (UID: "31b31734-c0f5-4653-9893-c3bb20fecb2c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 19:40:22.791317 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:22.790939    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret podName:d0da2917-eb23-47fc-9db2-dd7737386e59 nodeName:}" failed. No retries permitted until 2026-07-09 19:40:54.790931057 +0000 UTC m=+66.192065851 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret") pod "global-pull-secret-syncer-bjpjv" (UID: "d0da2917-eb23-47fc-9db2-dd7737386e59") : object "kube-system"/"original-pull-secret" not registered
Jul 09 19:40:23.071287 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:23.071215    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:23.071452 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:23.071215    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:23.071452 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:23.071347    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:23.071452 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:23.071397    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:23.071600 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:23.071517    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:23.071704 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:23.071674    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:25.071581 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:25.071545    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:25.072004 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:25.071545    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:25.072004 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:25.071701    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:25.072004 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:25.071551    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:25.072004 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:25.071795    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:25.072004 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:25.071871    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:27.071992 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:27.071956    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:27.071992 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:27.071994    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:27.072510 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:27.071956    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:27.072510 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:27.072071    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:27.072510 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:27.072147    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:27.072510 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:27.072241    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:27.267960 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:27.267926    2527 generic.go:356] "Generic (PLEG): container finished" podID="b7a72406-3b3d-4535-b5ba-8376627edc03" containerID="c07ac592bbc659c35fddd6f444844f2d13646979747f0dc78190199e3f557c85" exitCode=0
Jul 09 19:40:27.268111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:27.267978    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerDied","Data":"c07ac592bbc659c35fddd6f444844f2d13646979747f0dc78190199e3f557c85"}
Jul 09 19:40:28.273125 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:28.273094    2527 generic.go:356] "Generic (PLEG): container finished" podID="b7a72406-3b3d-4535-b5ba-8376627edc03" containerID="1dfb9be844f8360b6eec802f848f7a36043b8cddd94e7a87d41b94c78dbe7363" exitCode=0
Jul 09 19:40:28.273488 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:28.273153    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerDied","Data":"1dfb9be844f8360b6eec802f848f7a36043b8cddd94e7a87d41b94c78dbe7363"}
Jul 09 19:40:29.072593 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:29.072557    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:29.072786 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:29.072632    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:29.072786 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:29.072669    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:29.072786 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:29.072761    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:29.072786 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:29.072788    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:29.073028 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:29.072846    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:29.279036 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:29.279000    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" event={"ID":"b7a72406-3b3d-4535-b5ba-8376627edc03","Type":"ContainerStarted","Data":"722014ba177b2ca5ac322eedfb49f8738f5fa3a79b51a499c78ddafbcd4da05d"}
Jul 09 19:40:29.303961 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:29.303913    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9c2zf" podStartSLOduration=17.162584325 podStartE2EDuration="40.303898757s" totalImagesPullingTime="23.141314432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:29.302290147 +0000 UTC m=+40.703424957" watchObservedRunningTime="2026-07-09 19:40:29.303898757 +0000 UTC m=+40.705033567"
Jul 09 19:40:31.071903 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:31.071871    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:31.071903 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:31.071905    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:31.072335 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:31.071873    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:31.072335 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:31.072009    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:31.072335 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:31.072056    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:31.072335 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:31.072131    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:33.071731 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:33.071623    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:33.071731 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:33.071623    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:33.071731 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:33.071761    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:33.072373 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:33.071811    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:33.072373 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:33.071900    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:33.072373 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:33.071974    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:35.071975 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:35.071787    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:35.072407 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:35.071786    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:35.072407 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:35.072058    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:35.072407 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:35.071794    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:35.072407 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:35.072128    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:35.072407 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:35.072207    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:37.071608 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:37.071564    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:37.071608 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:37.071593    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:37.072067 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:37.071619    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:37.072067 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:37.071742    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:37.072067 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:37.071858    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:37.072067 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:37.071951    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:39.072127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:39.072095    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:39.072694 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:39.072184    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:39.072694 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:39.072296    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:39.072694 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:39.072424    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:39.072694 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:39.072473    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:39.072694 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:39.072532    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:41.071500 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:41.071467    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:41.071953 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:41.071468    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:41.071953 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:41.071578    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxkds" podUID="389b1ece-cc85-4eb3-9111-a57d697c98c2"
Jul 09 19:40:41.071953 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:41.071710    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjpjv" podUID="d0da2917-eb23-47fc-9db2-dd7737386e59"
Jul 09 19:40:41.071953 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:41.071468    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:41.071953 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:41.071845    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-42kwj" podUID="31b31734-c0f5-4653-9893-c3bb20fecb2c"
Jul 09 19:40:42.432195 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.432162    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-128.ec2.internal" event="NodeReady"
Jul 09 19:40:42.432578 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.432301    2527 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 19:40:42.470365 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.470333    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-594f6499b-nz69n"]
Jul 09 19:40:42.491163 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.491131    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-868wr"]
Jul 09 19:40:42.491318 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.491246    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.493925 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.493901    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 19:40:42.494060 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.494021    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rpggp\""
Jul 09 19:40:42.494260 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.494245    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 19:40:42.494303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.494277    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 19:40:42.499606 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.499588    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 19:40:42.514837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.514807    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-594f6499b-nz69n"]
Jul 09 19:40:42.514960 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.514848    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hqt7l"]
Jul 09 19:40:42.514960 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.514920    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.518133 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.518106    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 19:40:42.518133 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.518113    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dtfn7\""
Jul 09 19:40:42.518352 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.518340    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 19:40:42.538814 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.538783    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-868wr"]
Jul 09 19:40:42.538814 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.538821    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqt7l"]
Jul 09 19:40:42.538974 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.538898    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:42.541484 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.541460    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 19:40:42.541611 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.541559    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6qw7b\""
Jul 09 19:40:42.541611 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.541570    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 19:40:42.541888 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.541874    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 19:40:42.634986 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.634948    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-bound-sa-token\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635175 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635014    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e39e24-9e22-41ef-ba72-95c1c7490e92-cert\") pod \"ingress-canary-hqt7l\" (UID: \"c8e39e24-9e22-41ef-ba72-95c1c7490e92\") " pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:42.635175 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635042    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-image-registry-private-configuration\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635175 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635067    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-trusted-ca\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635175 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635140    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-registry-tls\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635175 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635173    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84d6705f-f43f-411c-8f1c-90b05200da73-metrics-tls\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635199    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnhz2\" (UniqueName: \"kubernetes.io/projected/84d6705f-f43f-411c-8f1c-90b05200da73-kube-api-access-wnhz2\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635221    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh5xp\" (UniqueName: \"kubernetes.io/projected/c8e39e24-9e22-41ef-ba72-95c1c7490e92-kube-api-access-wh5xp\") pod \"ingress-canary-hqt7l\" (UID: \"c8e39e24-9e22-41ef-ba72-95c1c7490e92\") " pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635239    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-installation-pull-secrets\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635257    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn4q9\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-kube-api-access-wn4q9\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635320    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-registry-certificates\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635346    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-ca-trust-extracted\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635363    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84d6705f-f43f-411c-8f1c-90b05200da73-config-volume\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.635383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.635378    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d6705f-f43f-411c-8f1c-90b05200da73-tmp-dir\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.735963 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.735868    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e39e24-9e22-41ef-ba72-95c1c7490e92-cert\") pod \"ingress-canary-hqt7l\" (UID: \"c8e39e24-9e22-41ef-ba72-95c1c7490e92\") " pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:42.735963 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.735916    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-image-registry-private-configuration\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.736137 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736034    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-trusted-ca\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.736137 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736101    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-registry-tls\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.736137 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736122    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84d6705f-f43f-411c-8f1c-90b05200da73-metrics-tls\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.736251 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736141    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnhz2\" (UniqueName: \"kubernetes.io/projected/84d6705f-f43f-411c-8f1c-90b05200da73-kube-api-access-wnhz2\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.736251 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736163    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh5xp\" (UniqueName: \"kubernetes.io/projected/c8e39e24-9e22-41ef-ba72-95c1c7490e92-kube-api-access-wh5xp\") pod \"ingress-canary-hqt7l\" (UID: \"c8e39e24-9e22-41ef-ba72-95c1c7490e92\") " pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:42.736251 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736193    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-installation-pull-secrets\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.736477 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736451    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn4q9\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-kube-api-access-wn4q9\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.736672 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736626    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-registry-certificates\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.736801 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736774    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-ca-trust-extracted\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.736801 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736809    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84d6705f-f43f-411c-8f1c-90b05200da73-config-volume\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.736965 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736832    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d6705f-f43f-411c-8f1c-90b05200da73-tmp-dir\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.736965 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.736867    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-bound-sa-token\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.737617 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.737120    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-trusted-ca\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.737617 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.737354    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-ca-trust-extracted\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.737617 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.737579    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84d6705f-f43f-411c-8f1c-90b05200da73-tmp-dir\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.737819 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.737670    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84d6705f-f43f-411c-8f1c-90b05200da73-config-volume\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.737819 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.737790    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-registry-certificates\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.740836 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.740804    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-image-registry-private-configuration\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.740836 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.740812    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-registry-tls\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.740964 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.740811    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84d6705f-f43f-411c-8f1c-90b05200da73-metrics-tls\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.740964 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.740813    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e39e24-9e22-41ef-ba72-95c1c7490e92-cert\") pod \"ingress-canary-hqt7l\" (UID: \"c8e39e24-9e22-41ef-ba72-95c1c7490e92\") " pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:42.740964 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.740888    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-installation-pull-secrets\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.744561 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.744540    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-bound-sa-token\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.744814 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.744797    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh5xp\" (UniqueName: \"kubernetes.io/projected/c8e39e24-9e22-41ef-ba72-95c1c7490e92-kube-api-access-wh5xp\") pod \"ingress-canary-hqt7l\" (UID: \"c8e39e24-9e22-41ef-ba72-95c1c7490e92\") " pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:42.745185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.745166    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnhz2\" (UniqueName: \"kubernetes.io/projected/84d6705f-f43f-411c-8f1c-90b05200da73-kube-api-access-wnhz2\") pod \"dns-default-868wr\" (UID: \"84d6705f-f43f-411c-8f1c-90b05200da73\") " pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.745278 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.745263    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn4q9\" (UniqueName: \"kubernetes.io/projected/ec51390a-ea19-4bda-a919-34fa8c5fbc8c-kube-api-access-wn4q9\") pod \"image-registry-594f6499b-nz69n\" (UID: \"ec51390a-ea19-4bda-a919-34fa8c5fbc8c\") " pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.801157 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.801114    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:42.823296 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.823269    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:42.847187 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:42.847160    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqt7l"
Jul 09 19:40:43.020469 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.020439    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-868wr"]
Jul 09 19:40:43.023449 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:43.023422    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d6705f_f43f_411c_8f1c_90b05200da73.slice/crio-2ee5b4dd5cf35040fc3fa241de1cda2e630b76e1345c7bd4a5c808c5c4999aa9 WatchSource:0}: Error finding container 2ee5b4dd5cf35040fc3fa241de1cda2e630b76e1345c7bd4a5c808c5c4999aa9: Status 404 returned error can't find the container with id 2ee5b4dd5cf35040fc3fa241de1cda2e630b76e1345c7bd4a5c808c5c4999aa9
Jul 09 19:40:43.027777 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.027755    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-594f6499b-nz69n"]
Jul 09 19:40:43.030537 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:43.030511    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec51390a_ea19_4bda_a919_34fa8c5fbc8c.slice/crio-5df20b66f51221725d60fe29303380ed9adda6467fb8ae6d611773cf351b7afe WatchSource:0}: Error finding container 5df20b66f51221725d60fe29303380ed9adda6467fb8ae6d611773cf351b7afe: Status 404 returned error can't find the container with id 5df20b66f51221725d60fe29303380ed9adda6467fb8ae6d611773cf351b7afe
Jul 09 19:40:43.038375 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.038352    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqt7l"]
Jul 09 19:40:43.041354 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:43.041329    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e39e24_9e22_41ef_ba72_95c1c7490e92.slice/crio-ef452f043ca818ba5c1a5de1b029050beda27bc99868e65c28b5629b358e0e7d WatchSource:0}: Error finding container ef452f043ca818ba5c1a5de1b029050beda27bc99868e65c28b5629b358e0e7d: Status 404 returned error can't find the container with id ef452f043ca818ba5c1a5de1b029050beda27bc99868e65c28b5629b358e0e7d
Jul 09 19:40:43.071193 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.071169    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:43.071288 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.071192    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:43.071288 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.071192    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:43.073804 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.073788    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 19:40:43.074179 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.074167    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-95dhd\""
Jul 09 19:40:43.074367 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.074354    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mczbz\""
Jul 09 19:40:43.074528 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.074514    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 19:40:43.075046 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.075032    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 19:40:43.075089 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.075057    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 19:40:43.316429 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.316398    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqt7l" event={"ID":"c8e39e24-9e22-41ef-ba72-95c1c7490e92","Type":"ContainerStarted","Data":"ef452f043ca818ba5c1a5de1b029050beda27bc99868e65c28b5629b358e0e7d"}
Jul 09 19:40:43.317565 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.317546    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-868wr" event={"ID":"84d6705f-f43f-411c-8f1c-90b05200da73","Type":"ContainerStarted","Data":"2ee5b4dd5cf35040fc3fa241de1cda2e630b76e1345c7bd4a5c808c5c4999aa9"}
Jul 09 19:40:43.319107 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.319077    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-594f6499b-nz69n" event={"ID":"ec51390a-ea19-4bda-a919-34fa8c5fbc8c","Type":"ContainerStarted","Data":"0c8d675c28ee865ba5021de6c9a5d787d6115bdb62467be92528eb7e2f2ffda2"}
Jul 09 19:40:43.319188 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.319119    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-594f6499b-nz69n" event={"ID":"ec51390a-ea19-4bda-a919-34fa8c5fbc8c","Type":"ContainerStarted","Data":"5df20b66f51221725d60fe29303380ed9adda6467fb8ae6d611773cf351b7afe"}
Jul 09 19:40:43.319241 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.319219    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:40:43.350513 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:43.350454    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-594f6499b-nz69n" podStartSLOduration=2.350437316 podStartE2EDuration="2.350437316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:43.345403928 +0000 UTC m=+54.746538737" watchObservedRunningTime="2026-07-09 19:40:43.350437316 +0000 UTC m=+54.751572153"
Jul 09 19:40:45.361968 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.361937    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67f979994b-qqvbg"]
Jul 09 19:40:45.392134 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.392085    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f979994b-qqvbg"]
Jul 09 19:40:45.392301 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.392192    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.394971 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.394938    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 19:40:45.396544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.396103    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 19:40:45.396544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.396103    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wztb4\""
Jul 09 19:40:45.396544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.396153    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 19:40:45.396544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.396175    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 19:40:45.396544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.396150    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 19:40:45.396544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.396288    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 19:40:45.396544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.396425    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 19:40:45.561943 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.561910    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-service-ca\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.562127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.561960    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-oauth-config\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.562127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.561987    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2dt9\" (UniqueName: \"kubernetes.io/projected/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-kube-api-access-b2dt9\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.562127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.562093    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-oauth-serving-cert\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.562253 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.562144    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-serving-cert\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.562287 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.562271    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-config\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.663086 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.662990    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-oauth-serving-cert\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.663086 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.663077    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-serving-cert\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.663310 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.663133    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-config\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.663310 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.663158    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-service-ca\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.663310 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.663188    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-oauth-config\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.663310 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.663212    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2dt9\" (UniqueName: \"kubernetes.io/projected/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-kube-api-access-b2dt9\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.663789 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.663757    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-oauth-serving-cert\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.664129 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.664087    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-service-ca\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.664307 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.664285    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-config\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.666107 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.666084    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-serving-cert\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.666543 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.666524    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-oauth-config\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.687592 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.687505    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2dt9\" (UniqueName: \"kubernetes.io/projected/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-kube-api-access-b2dt9\") pod \"console-67f979994b-qqvbg\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") " pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:45.702386 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:45.702343    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:46.179365 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:46.179338    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f979994b-qqvbg"]
Jul 09 19:40:46.182179 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:46.182153    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe56dd1c_2ce7_4082_8cfa_5055f692f40c.slice/crio-5ca9548d684dd63a6f08a5e19ab50d4be52a1f3feab316e59631a2c5485b650e WatchSource:0}: Error finding container 5ca9548d684dd63a6f08a5e19ab50d4be52a1f3feab316e59631a2c5485b650e: Status 404 returned error can't find the container with id 5ca9548d684dd63a6f08a5e19ab50d4be52a1f3feab316e59631a2c5485b650e
Jul 09 19:40:46.327956 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:46.327872    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-868wr" event={"ID":"84d6705f-f43f-411c-8f1c-90b05200da73","Type":"ContainerStarted","Data":"9745cd8f581a1d975b8f11e75b7e3b27b30ecbb60769b9c5380854c5734cc0a6"}
Jul 09 19:40:46.329113 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:46.329089    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f979994b-qqvbg" event={"ID":"fe56dd1c-2ce7-4082-8cfa-5055f692f40c","Type":"ContainerStarted","Data":"5ca9548d684dd63a6f08a5e19ab50d4be52a1f3feab316e59631a2c5485b650e"}
Jul 09 19:40:47.334657 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.334602    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-868wr" event={"ID":"84d6705f-f43f-411c-8f1c-90b05200da73","Type":"ContainerStarted","Data":"b521299fa1a98a450b3765e1162cae9bc8bede5933d939cfff67b59f7eab0358"}
Jul 09 19:40:47.335116 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.334737    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:47.336665 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.336613    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqt7l" event={"ID":"c8e39e24-9e22-41ef-ba72-95c1c7490e92","Type":"ContainerStarted","Data":"e7e1a1218f51b19959ff5fba3f77c7289d9fa3a5ce40d18fc3195b7d85a1f4e6"}
Jul 09 19:40:47.356940 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.356880    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-868wr" podStartSLOduration=2.34001174 podStartE2EDuration="5.356859977s" totalImagesPullingTime="3.016848237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:47.355040874 +0000 UTC m=+58.756175723" watchObservedRunningTime="2026-07-09 19:40:47.356859977 +0000 UTC m=+58.757994788"
Jul 09 19:40:47.371984 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.371940    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hqt7l" podStartSLOduration=2.048954156 podStartE2EDuration="5.371924956s" totalImagesPullingTime="3.3229708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:47.371100726 +0000 UTC m=+58.772235538" watchObservedRunningTime="2026-07-09 19:40:47.371924956 +0000 UTC m=+58.773059766"
Jul 09 19:40:47.730696 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.730605    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-968dc5c8f-lnln4"]
Jul 09 19:40:47.754984 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.754947    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-968dc5c8f-lnln4"]
Jul 09 19:40:47.755129 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.755042    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.766127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.766097    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 19:40:47.780289 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.780253    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-service-ca\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.780289 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.780303    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-oauth-config\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.780517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.780338    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-serving-cert\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.780517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.780387    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v6jp\" (UniqueName: \"kubernetes.io/projected/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-kube-api-access-4v6jp\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.780517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.780425    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-oauth-serving-cert\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.780517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.780446    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-config\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.780517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.780466    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-trusted-ca-bundle\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.881800 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.881757    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-serving-cert\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.881988 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.881843    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v6jp\" (UniqueName: \"kubernetes.io/projected/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-kube-api-access-4v6jp\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.881988 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.881893    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-oauth-serving-cert\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.881988 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.881919    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-config\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.881988 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.881945    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-trusted-ca-bundle\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.882202 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.882004    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-service-ca\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.882202 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.882029    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-oauth-config\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.882840 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.882808    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-config\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.882961 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.882896    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-oauth-serving-cert\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.883027 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.883012    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-service-ca\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.883076 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.883035    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-trusted-ca-bundle\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.884583 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.884559    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-oauth-config\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.884729 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.884704    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-serving-cert\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:47.890947 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:47.890922    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v6jp\" (UniqueName: \"kubernetes.io/projected/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-kube-api-access-4v6jp\") pod \"console-968dc5c8f-lnln4\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") " pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:48.066088 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:48.066050    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:48.245793 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:48.245756    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-968dc5c8f-lnln4"]
Jul 09 19:40:48.251182 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:48.250686    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a7fa4f_7d2d_4e42_8ba1_0989aad24604.slice/crio-e5fed104adf4ee682664416498c938564d6c849ca9f90665441b957e0cd5e001 WatchSource:0}: Error finding container e5fed104adf4ee682664416498c938564d6c849ca9f90665441b957e0cd5e001: Status 404 returned error can't find the container with id e5fed104adf4ee682664416498c938564d6c849ca9f90665441b957e0cd5e001
Jul 09 19:40:48.341283 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:48.341159    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968dc5c8f-lnln4" event={"ID":"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604","Type":"ContainerStarted","Data":"e5fed104adf4ee682664416498c938564d6c849ca9f90665441b957e0cd5e001"}
Jul 09 19:40:50.141336 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.141247    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"]
Jul 09 19:40:50.166051 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.166021    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"]
Jul 09 19:40:50.166203 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.166130    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:50.169054 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.169031    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mkrwn\""
Jul 09 19:40:50.169193 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.169031    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 19:40:50.201578 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.201544    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e74eff0b-696b-4449-89f2-efec0718a154-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-78z72\" (UID: \"e74eff0b-696b-4449-89f2-efec0718a154\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:50.302105 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.302067    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e74eff0b-696b-4449-89f2-efec0718a154-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-78z72\" (UID: \"e74eff0b-696b-4449-89f2-efec0718a154\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:50.302271 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:50.302219    2527 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 09 19:40:50.302311 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:50.302286    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74eff0b-696b-4449-89f2-efec0718a154-tls-certificates podName:e74eff0b-696b-4449-89f2-efec0718a154 nodeName:}" failed. No retries permitted until 2026-07-09 19:40:50.802271863 +0000 UTC m=+62.203406651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/e74eff0b-696b-4449-89f2-efec0718a154-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-78z72" (UID: "e74eff0b-696b-4449-89f2-efec0718a154") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 09 19:40:50.349463 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.349429    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f979994b-qqvbg" event={"ID":"fe56dd1c-2ce7-4082-8cfa-5055f692f40c","Type":"ContainerStarted","Data":"97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0"}
Jul 09 19:40:50.350955 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.350931    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968dc5c8f-lnln4" event={"ID":"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604","Type":"ContainerStarted","Data":"bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b"}
Jul 09 19:40:50.390704 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.390526    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67f979994b-qqvbg" podStartSLOduration=1.622287532 podStartE2EDuration="5.390508578s" totalImagesPullingTime="3.768221046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:50.371288407 +0000 UTC m=+61.772423220" watchObservedRunningTime="2026-07-09 19:40:50.390508578 +0000 UTC m=+61.791643388"
Jul 09 19:40:50.390859 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.390771    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-968dc5c8f-lnln4" podStartSLOduration=1.69066269 podStartE2EDuration="3.390761008s" totalImagesPullingTime="1.700098318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:50.389921801 +0000 UTC m=+61.791056611" watchObservedRunningTime="2026-07-09 19:40:50.390761008 +0000 UTC m=+61.791895808"
Jul 09 19:40:50.740879 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.740843    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vmfjg"]
Jul 09 19:40:50.759890 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.759859    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.761371 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.761339    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vmfjg"]
Jul 09 19:40:50.762628 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.762607    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 19:40:50.762941 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.762921    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 19:40:50.763155 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.763137    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 19:40:50.763276 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.763256    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hw6z8\""
Jul 09 19:40:50.763888 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.763871    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 19:40:50.806323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.806292    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7ckz\" (UniqueName: \"kubernetes.io/projected/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-api-access-x7ckz\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.806488 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.806350    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-data-volume\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.806488 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.806394    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.806488 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.806467    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e74eff0b-696b-4449-89f2-efec0718a154-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-78z72\" (UID: \"e74eff0b-696b-4449-89f2-efec0718a154\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:50.806693 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.806492    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-crio-socket\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.806693 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.806519    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.809104 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.809079    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e74eff0b-696b-4449-89f2-efec0718a154-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-78z72\" (UID: \"e74eff0b-696b-4449-89f2-efec0718a154\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:50.907521 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.907488    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-data-volume\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.907521 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.907528    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.907561    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-crio-socket\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.907586    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:50.907627    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.907641    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7ckz\" (UniqueName: \"kubernetes.io/projected/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-api-access-x7ckz\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:50.907722    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:51.407705617 +0000 UTC m=+62.808840405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : configmap "kube-rbac-proxy" not found
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.907737    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-crio-socket\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:50.907766    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:50.907815 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:50.907821    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:51.407804772 +0000 UTC m=+62.808939563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:50.908149 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.907977    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-data-volume\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:50.916420 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:50.916400    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7ckz\" (UniqueName: \"kubernetes.io/projected/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-api-access-x7ckz\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:51.075542 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:51.075515    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:51.198816 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:51.198786    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"]
Jul 09 19:40:51.201954 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:51.201927    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74eff0b_696b_4449_89f2_efec0718a154.slice/crio-8d1ba8f20c90f523bd9ed526d181882b89e7c3bc02c90f351f2619f6b69e3f29 WatchSource:0}: Error finding container 8d1ba8f20c90f523bd9ed526d181882b89e7c3bc02c90f351f2619f6b69e3f29: Status 404 returned error can't find the container with id 8d1ba8f20c90f523bd9ed526d181882b89e7c3bc02c90f351f2619f6b69e3f29
Jul 09 19:40:51.262408 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:51.262382    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlhxk"
Jul 09 19:40:51.355975 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:51.355882    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72" event={"ID":"e74eff0b-696b-4449-89f2-efec0718a154","Type":"ContainerStarted","Data":"8d1ba8f20c90f523bd9ed526d181882b89e7c3bc02c90f351f2619f6b69e3f29"}
Jul 09 19:40:51.412279 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:51.412242    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:51.412533 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:51.412509    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:40:51.412634 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:51.412576    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:52.412558063 +0000 UTC m=+63.813692856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : configmap "kube-rbac-proxy" not found
Jul 09 19:40:51.412748 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:51.412689    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:51.412807 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:51.412770    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:51.412851 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:51.412822    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:52.412805797 +0000 UTC m=+63.813940589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:52.419701 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:52.419668    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:52.420078 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:52.419732    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:52.420078 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:52.419804    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:40:52.420078 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:52.419872    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:52.420078 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:52.419872    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:54.419852979 +0000 UTC m=+65.820987768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : configmap "kube-rbac-proxy" not found
Jul 09 19:40:52.420078 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:52.419914    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:54.419903791 +0000 UTC m=+65.821038579 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:53.140554 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.140525    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67f979994b-qqvbg_fe56dd1c-2ce7-4082-8cfa-5055f692f40c/console/0.log"
Jul 09 19:40:53.325208 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.325181    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-968dc5c8f-lnln4_f4a7fa4f-7d2d-4e42-8ba1-0989aad24604/console/0.log"
Jul 09 19:40:53.364151 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.364110    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72" event={"ID":"e74eff0b-696b-4449-89f2-efec0718a154","Type":"ContainerStarted","Data":"6d2a3735ac7a47d14fa9ad26f9bf50cc13eb141553d7c2739e07912ba5d64eb8"}
Jul 09 19:40:53.364351 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.364330    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:53.368944 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.368922    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72"
Jul 09 19:40:53.396736 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.396629    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-78z72" podStartSLOduration=2.269995571 podStartE2EDuration="3.396610511s" totalImagesPullingTime="1.12661494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:53.380160371 +0000 UTC m=+64.781295180" watchObservedRunningTime="2026-07-09 19:40:53.396610511 +0000 UTC m=+64.797745322"
Jul 09 19:40:53.725067 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.724969    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-868wr_84d6705f-f43f-411c-8f1c-90b05200da73/dns/0.log"
Jul 09 19:40:53.925267 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:53.925216    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-868wr_84d6705f-f43f-411c-8f1c-90b05200da73/kube-rbac-proxy/0.log"
Jul 09 19:40:54.211876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.211845    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-5sxjl"]
Jul 09 19:40:54.214924 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.214908    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.218904 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.218868    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 19:40:54.219038 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.218868    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 19:40:54.219038 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.218879    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-v5vc8\""
Jul 09 19:40:54.219038 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.218917    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 19:40:54.219038 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.219012    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 19:40:54.219038 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.218920    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 19:40:54.225474 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.225442    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-5sxjl"]
Jul 09 19:40:54.335714 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.335671    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cc7e313-6b65-42d0-b0e1-13957d12977f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.335896 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.335810    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf9cf\" (UniqueName: \"kubernetes.io/projected/2cc7e313-6b65-42d0-b0e1-13957d12977f-kube-api-access-jf9cf\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.335896 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.335847    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.335896 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.335892    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.436214 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.436179    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:54.436387 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.436247    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf9cf\" (UniqueName: \"kubernetes.io/projected/2cc7e313-6b65-42d0-b0e1-13957d12977f-kube-api-access-jf9cf\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.436387 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.436268    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.436387 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.436296    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:54.436387 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.436315    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.436387 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:54.436341    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:54.436387 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.436353    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cc7e313-6b65-42d0-b0e1-13957d12977f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.436759 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:54.436420    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:58.436399732 +0000 UTC m=+69.837534520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:54.436759 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:54.436452    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:40:54.436759 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:54.436522    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:40:58.43650684 +0000 UTC m=+69.837641632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : configmap "kube-rbac-proxy" not found
Jul 09 19:40:54.436759 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:54.436460    2527 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 09 19:40:54.436759 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:54.436573    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-tls podName:2cc7e313-6b65-42d0-b0e1-13957d12977f nodeName:}" failed. No retries permitted until 2026-07-09 19:40:54.936562249 +0000 UTC m=+66.337697053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-tls") pod "prometheus-operator-64785b6675-5sxjl" (UID: "2cc7e313-6b65-42d0-b0e1-13957d12977f") : secret "prometheus-operator-tls" not found
Jul 09 19:40:54.437114 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.437097    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2cc7e313-6b65-42d0-b0e1-13957d12977f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.438836 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.438815    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.445049 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.445027    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf9cf\" (UniqueName: \"kubernetes.io/projected/2cc7e313-6b65-42d0-b0e1-13957d12977f-kube-api-access-jf9cf\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.839392 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.839360    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:54.839866 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.839475    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:54.839866 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.839515    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:54.842341 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.842324    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 19:40:54.842392 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.842324    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 19:40:54.842392 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.842324    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 19:40:54.851968 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.851945    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 19:40:54.852096 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.851967    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31b31734-c0f5-4653-9893-c3bb20fecb2c-metrics-certs\") pod \"network-metrics-daemon-42kwj\" (UID: \"31b31734-c0f5-4653-9893-c3bb20fecb2c\") " pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:54.852096 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.852018    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0da2917-eb23-47fc-9db2-dd7737386e59-original-pull-secret\") pod \"global-pull-secret-syncer-bjpjv\" (UID: \"d0da2917-eb23-47fc-9db2-dd7737386e59\") " pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:54.862616 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.862593    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtgsh\" (UniqueName: \"kubernetes.io/projected/389b1ece-cc85-4eb3-9111-a57d697c98c2-kube-api-access-rtgsh\") pod \"network-check-target-zxkds\" (UID: \"389b1ece-cc85-4eb3-9111-a57d697c98c2\") " pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:54.939890 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.939853    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:54.942233 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:54.942203    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2cc7e313-6b65-42d0-b0e1-13957d12977f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-5sxjl\" (UID: \"2cc7e313-6b65-42d0-b0e1-13957d12977f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:55.088953 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.088918    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-95dhd\""
Jul 09 19:40:55.097060 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.096993    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:40:55.101986 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.101967    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjpjv"
Jul 09 19:40:55.109296 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.109269    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mczbz\""
Jul 09 19:40:55.117579 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.117557    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-42kwj"
Jul 09 19:40:55.123681 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.123629    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl"
Jul 09 19:40:55.262024 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.258872    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bjpjv"]
Jul 09 19:40:55.284040 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.283996    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zxkds"]
Jul 09 19:40:55.285609 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:55.285531    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389b1ece_cc85_4eb3_9111_a57d697c98c2.slice/crio-f6f567764f725cb6611daecd0a52c6d6369cff2bb20832efc679b722533fb893 WatchSource:0}: Error finding container f6f567764f725cb6611daecd0a52c6d6369cff2bb20832efc679b722533fb893: Status 404 returned error can't find the container with id f6f567764f725cb6611daecd0a52c6d6369cff2bb20832efc679b722533fb893
Jul 09 19:40:55.300968 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.300944    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-42kwj"]
Jul 09 19:40:55.303239 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:55.303200    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b31734_c0f5_4653_9893_c3bb20fecb2c.slice/crio-de95535d2180a256f42db3762949ea63970cbb5b514c008093eb3e104edd1cca WatchSource:0}: Error finding container de95535d2180a256f42db3762949ea63970cbb5b514c008093eb3e104edd1cca: Status 404 returned error can't find the container with id de95535d2180a256f42db3762949ea63970cbb5b514c008093eb3e104edd1cca
Jul 09 19:40:55.319082 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.319048    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-5sxjl"]
Jul 09 19:40:55.322012 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:40:55.321990    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc7e313_6b65_42d0_b0e1_13957d12977f.slice/crio-bb9bfe2cfa241fb43384057e75d5da9a6e9229c00a089b2a997d0b95ea2dabbe WatchSource:0}: Error finding container bb9bfe2cfa241fb43384057e75d5da9a6e9229c00a089b2a997d0b95ea2dabbe: Status 404 returned error can't find the container with id bb9bfe2cfa241fb43384057e75d5da9a6e9229c00a089b2a997d0b95ea2dabbe
Jul 09 19:40:55.325482 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.325465    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sd999_9dadcbf5-686f-4448-ab73-d05ec262206a/dns-node-resolver/0.log"
Jul 09 19:40:55.371513 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.371417    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zxkds" event={"ID":"389b1ece-cc85-4eb3-9111-a57d697c98c2","Type":"ContainerStarted","Data":"f6f567764f725cb6611daecd0a52c6d6369cff2bb20832efc679b722533fb893"}
Jul 09 19:40:55.372484 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.372460    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl" event={"ID":"2cc7e313-6b65-42d0-b0e1-13957d12977f","Type":"ContainerStarted","Data":"bb9bfe2cfa241fb43384057e75d5da9a6e9229c00a089b2a997d0b95ea2dabbe"}
Jul 09 19:40:55.373440 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.373423    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-42kwj" event={"ID":"31b31734-c0f5-4653-9893-c3bb20fecb2c","Type":"ContainerStarted","Data":"de95535d2180a256f42db3762949ea63970cbb5b514c008093eb3e104edd1cca"}
Jul 09 19:40:55.374388 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.374370    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bjpjv" event={"ID":"d0da2917-eb23-47fc-9db2-dd7737386e59","Type":"ContainerStarted","Data":"4b37f9b47451da9dcbe2a911927e82a1b68a5ff9f76811334f8dd2f51d00bf11"}
Jul 09 19:40:55.524450 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.524422    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-594f6499b-nz69n_ec51390a-ea19-4bda-a919-34fa8c5fbc8c/registry/0.log"
Jul 09 19:40:55.702750 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.702658    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:55.702750 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.702726    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:40:55.704056 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.704028    2527 patch_prober.go:28] interesting pod/console-67f979994b-qqvbg container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 09 19:40:55.704164 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:55.704075    2527 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-67f979994b-qqvbg" podUID="fe56dd1c-2ce7-4082-8cfa-5055f692f40c" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 09 19:40:56.124955 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:56.124924    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nfv4x_ea4bda2e-4a85-4410-88a0-843cae158512/node-ca/0.log"
Jul 09 19:40:56.924701 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:56.924664    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqt7l_c8e39e24-9e22-41ef-ba72-95c1c7490e92/serve-healthcheck-canary/0.log"
Jul 09 19:40:57.344008 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:57.343977    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-868wr"
Jul 09 19:40:57.386524 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:57.386489    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl" event={"ID":"2cc7e313-6b65-42d0-b0e1-13957d12977f","Type":"ContainerStarted","Data":"7510884b9f5fea30486bff7b079b57b168fee818b8b71582db7afde97d07c9b5"}
Jul 09 19:40:57.386717 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:57.386557    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl" event={"ID":"2cc7e313-6b65-42d0-b0e1-13957d12977f","Type":"ContainerStarted","Data":"e44ef1b811df188f4125c9da60a1a9ad4ebd9737343bdf15088fc14251556dde"}
Jul 09 19:40:57.389208 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:57.389176    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-42kwj" event={"ID":"31b31734-c0f5-4653-9893-c3bb20fecb2c","Type":"ContainerStarted","Data":"3e86a0fcdcbcae4f90dbc304b7b1a138ee9ea848eddf77f8a9b0a9770d67834e"}
Jul 09 19:40:57.389337 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:57.389240    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-42kwj" event={"ID":"31b31734-c0f5-4653-9893-c3bb20fecb2c","Type":"ContainerStarted","Data":"787a47c74bc1c8523491fb6f8643dfe639d5a6421bc837367fb8b006c7df3734"}
Jul 09 19:40:57.426224 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:57.426163    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-42kwj" podStartSLOduration=66.745521773 podStartE2EDuration="1m8.426141125s" totalImagesPullingTime="1.680619352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:57.425937139 +0000 UTC m=+68.827071950" watchObservedRunningTime="2026-07-09 19:40:57.426141125 +0000 UTC m=+68.827275938"
Jul 09 19:40:57.426393 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:57.426352    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-5sxjl" podStartSLOduration=1.7632040390000001 podStartE2EDuration="3.426343881s" totalImagesPullingTime="1.663139842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:57.408292601 +0000 UTC m=+68.809427406" watchObservedRunningTime="2026-07-09 19:40:57.426343881 +0000 UTC m=+68.827478693"
Jul 09 19:40:58.066579 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:58.066542    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:58.066780 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:58.066670    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:40:58.068155 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:58.068117    2527 patch_prober.go:28] interesting pod/console-968dc5c8f-lnln4 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused" start-of-body=
Jul 09 19:40:58.068271 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:58.068196    2527 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-968dc5c8f-lnln4" podUID="f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" containerName="console" probeResult="failure" output="Get \"https://10.134.0.10:8443/health\": dial tcp 10.134.0.10:8443: connect: connection refused"
Jul 09 19:40:58.469956 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:58.469874    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:58.470302 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:58.470040    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:40:58.470302 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:58.470095    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:40:58.470302 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:58.470149    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:41:06.470133816 +0000 UTC m=+77.871268604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : configmap "kube-rbac-proxy" not found
Jul 09 19:40:58.470492 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:58.470474    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:58.470547 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:58.470520    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls podName:4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc nodeName:}" failed. No retries permitted until 2026-07-09 19:41:06.470507219 +0000 UTC m=+77.871642008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmfjg" (UID: "4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:40:59.567339 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.567300    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"]
Jul 09 19:40:59.570750 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.570728    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.573534 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.573511    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 19:40:59.573534 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.573531    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 19:40:59.573729 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.573587    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-djplm\""
Jul 09 19:40:59.583968 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.583945    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"]
Jul 09 19:40:59.610519 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.610490    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hg4ml"]
Jul 09 19:40:59.613964 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.613938    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"]
Jul 09 19:40:59.614110 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.614070    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.616885 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.616864    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 19:40:59.617173 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.617152    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.617566 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.617545    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 19:40:59.617821 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.617687    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wcdj6\""
Jul 09 19:40:59.617900 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.617830    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 19:40:59.620787 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.620765    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j9g27\""
Jul 09 19:40:59.620873 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.620810    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 19:40:59.620928 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.620897    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 19:40:59.621013 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.620997    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 19:40:59.628234 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.628212    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"]
Jul 09 19:40:59.684016 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.683979    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-textfile\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684203 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684026    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.684203 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684110    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f31eedf-1c5a-4825-b852-3b66d081e2c1-metrics-client-ca\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684203 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684161    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a05fdd56-7577-42c5-8345-1f4d1730331c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.684334 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684214    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-sys\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684334 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684241    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgx8c\" (UniqueName: \"kubernetes.io/projected/a05fdd56-7577-42c5-8345-1f4d1730331c-kube-api-access-zgx8c\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.684334 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684283    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-wtmp\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684334 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684307    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.684495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684344    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-root\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684374    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.684495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684401    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.684495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684433    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-tls\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684462    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684491    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a05fdd56-7577-42c5-8345-1f4d1730331c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.684771 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684519    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a05fdd56-7577-42c5-8345-1f4d1730331c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.684771 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684564    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5m99\" (UniqueName: \"kubernetes.io/projected/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-api-access-k5m99\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.684771 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684593    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684771 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684617    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqp4z\" (UniqueName: \"kubernetes.io/projected/1f31eedf-1c5a-4825-b852-3b66d081e2c1-kube-api-access-tqp4z\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.684771 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.684685    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.786127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786081    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.786127 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786125    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.786352 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786155    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-tls\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.786352 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786177    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.786352 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786195    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a05fdd56-7577-42c5-8345-1f4d1730331c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.786352 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786214    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a05fdd56-7577-42c5-8345-1f4d1730331c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.786352 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:59.786279    2527 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 19:40:59.786591 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:40:59.786369    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-tls podName:1f31eedf-1c5a-4825-b852-3b66d081e2c1 nodeName:}" failed. No retries permitted until 2026-07-09 19:41:00.286346512 +0000 UTC m=+71.687481319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-tls") pod "node-exporter-hg4ml" (UID: "1f31eedf-1c5a-4825-b852-3b66d081e2c1") : secret "node-exporter-tls" not found
Jul 09 19:40:59.786693 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786613    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5m99\" (UniqueName: \"kubernetes.io/projected/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-api-access-k5m99\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.786693 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786682    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.786799 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786709    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqp4z\" (UniqueName: \"kubernetes.io/projected/1f31eedf-1c5a-4825-b852-3b66d081e2c1-kube-api-access-tqp4z\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.786799 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786734    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.786799 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786779    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-textfile\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.786943 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786810    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.786943 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786843    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f31eedf-1c5a-4825-b852-3b66d081e2c1-metrics-client-ca\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.786943 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786874    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a05fdd56-7577-42c5-8345-1f4d1730331c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.786943 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786919    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-sys\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786944    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgx8c\" (UniqueName: \"kubernetes.io/projected/a05fdd56-7577-42c5-8345-1f4d1730331c-kube-api-access-zgx8c\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.787303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.786984    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-wtmp\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787010    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.787303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787050    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-root\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787059    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.787303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787140    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-root\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787303 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787255    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.787636 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787369    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-sys\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787636 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787388    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-textfile\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787636 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787547    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787636 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787616    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-wtmp\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.787866 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.787843    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f31eedf-1c5a-4825-b852-3b66d081e2c1-metrics-client-ca\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.788134 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.788111    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.788746 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.788719    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a05fdd56-7577-42c5-8345-1f4d1730331c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.789294 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.789267    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.789493 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.789463    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a05fdd56-7577-42c5-8345-1f4d1730331c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.790269 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.790247    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.790353 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.790287    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a05fdd56-7577-42c5-8345-1f4d1730331c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.791312 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.791289    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.803185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.803165    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5m99\" (UniqueName: \"kubernetes.io/projected/a3fd7f1e-8d3e-4cf1-bc97-1975001adaad-kube-api-access-k5m99\") pod \"kube-state-metrics-785778ff74-f5t7k\" (UID: \"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:40:59.803487 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.803462    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqp4z\" (UniqueName: \"kubernetes.io/projected/1f31eedf-1c5a-4825-b852-3b66d081e2c1-kube-api-access-tqp4z\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:40:59.804229 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.804205    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgx8c\" (UniqueName: \"kubernetes.io/projected/a05fdd56-7577-42c5-8345-1f4d1730331c-kube-api-access-zgx8c\") pod \"openshift-state-metrics-84bd68b896-9657v\" (UID: \"a05fdd56-7577-42c5-8345-1f4d1730331c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.880675 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.880571    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"
Jul 09 19:40:59.933480 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:40:59.933250    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"
Jul 09 19:41:00.292756 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.292721    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-tls\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:41:00.295516 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.295489    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f31eedf-1c5a-4825-b852-3b66d081e2c1-node-exporter-tls\") pod \"node-exporter-hg4ml\" (UID: \"1f31eedf-1c5a-4825-b852-3b66d081e2c1\") " pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:41:00.526243 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.526205    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hg4ml"
Jul 09 19:41:00.600012 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:00.599979    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f31eedf_1c5a_4825_b852_3b66d081e2c1.slice/crio-bda44d348e4eec09d36006af6210fb7d4a3bb28457b2826cbb6ddcfa9e1d91a0 WatchSource:0}: Error finding container bda44d348e4eec09d36006af6210fb7d4a3bb28457b2826cbb6ddcfa9e1d91a0: Status 404 returned error can't find the container with id bda44d348e4eec09d36006af6210fb7d4a3bb28457b2826cbb6ddcfa9e1d91a0
Jul 09 19:41:00.690201 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.690166    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 19:41:00.694567 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.694480    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.699296 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.699069    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 19:41:00.699493 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.699370    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.699819    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.700101    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.700324    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.700547    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.701175    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.701385    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8wckx\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.701594    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 19:41:00.702186 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.701839    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 19:41:00.716427 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.716303    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 19:41:00.762530 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.762506    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-f5t7k"]
Jul 09 19:41:00.764724 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:00.764699    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3fd7f1e_8d3e_4cf1_bc97_1975001adaad.slice/crio-db1abb1ff77bcb2db23e9cedbabcba213b23d8e676bf3571c7352a0f1414feed WatchSource:0}: Error finding container db1abb1ff77bcb2db23e9cedbabcba213b23d8e676bf3571c7352a0f1414feed: Status 404 returned error can't find the container with id db1abb1ff77bcb2db23e9cedbabcba213b23d8e676bf3571c7352a0f1414feed
Jul 09 19:41:00.787802 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.787774    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9657v"]
Jul 09 19:41:00.796993 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.796895    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3f3fc16d-d073-41d8-8f21-b07f370fea0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.796993 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.796924    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.796993 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.796942    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.796997    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqr2h\" (UniqueName: \"kubernetes.io/projected/3f3fc16d-d073-41d8-8f21-b07f370fea0c-kube-api-access-rqr2h\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797029    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797048    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797079    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797100    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3f3fc16d-d073-41d8-8f21-b07f370fea0c-config-out\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797149    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797471 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797189    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797471 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797222    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797471 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797254    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.797471 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.797354    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-web-config\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898038 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898005    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-web-config\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898179 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898046    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3f3fc16d-d073-41d8-8f21-b07f370fea0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898179 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898077    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898179 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898132    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898179 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:00.898164    2527 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898178    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqr2h\" (UniqueName: \"kubernetes.io/projected/3f3fc16d-d073-41d8-8f21-b07f370fea0c-kube-api-access-rqr2h\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:00.898211    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-main-tls podName:3f3fc16d-d073-41d8-8f21-b07f370fea0c nodeName:}" failed. No retries permitted until 2026-07-09 19:41:01.398197955 +0000 UTC m=+72.799332742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "3f3fc16d-d073-41d8-8f21-b07f370fea0c") : secret "alertmanager-main-tls" not found
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898223    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898238    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898257    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898274    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3f3fc16d-d073-41d8-8f21-b07f370fea0c-config-out\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898299    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898328    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898346    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898798 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.898391    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.898798 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:00.898559    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-trusted-ca-bundle podName:3f3fc16d-d073-41d8-8f21-b07f370fea0c nodeName:}" failed. No retries permitted until 2026-07-09 19:41:01.398541434 +0000 UTC m=+72.799676251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "3f3fc16d-d073-41d8-8f21-b07f370fea0c") : configmap references non-existent config key: ca-bundle.crt
Jul 09 19:41:00.902739 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.901230    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3f3fc16d-d073-41d8-8f21-b07f370fea0c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.902739 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.901499    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.902739 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.901956    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.903193 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.903171    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3f3fc16d-d073-41d8-8f21-b07f370fea0c-config-out\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.904031 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.903979    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-web-config\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.904386 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.904276    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.904827 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.904805    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.905214 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.905192    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.905825 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.905781    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.907532 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.907507    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-config-volume\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:00.908987 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:00.908955    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqr2h\" (UniqueName: \"kubernetes.io/projected/3f3fc16d-d073-41d8-8f21-b07f370fea0c-kube-api-access-rqr2h\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:01.402633 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.402599    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:01.402826 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.402640    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:01.403570 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.403535    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f3fc16d-d073-41d8-8f21-b07f370fea0c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:01.405987 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.405965    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3f3fc16d-d073-41d8-8f21-b07f370fea0c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3f3fc16d-d073-41d8-8f21-b07f370fea0c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:01.406480 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.406444    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bjpjv" event={"ID":"d0da2917-eb23-47fc-9db2-dd7737386e59","Type":"ContainerStarted","Data":"1a3faae4fd800b2b59a3b4feabbc609e7adcdd216c4030ff5c2c97df428afbab"}
Jul 09 19:41:01.407811 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.407787    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k" event={"ID":"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad","Type":"ContainerStarted","Data":"db1abb1ff77bcb2db23e9cedbabcba213b23d8e676bf3571c7352a0f1414feed"}
Jul 09 19:41:01.409045 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.409022    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg4ml" event={"ID":"1f31eedf-1c5a-4825-b852-3b66d081e2c1","Type":"ContainerStarted","Data":"bda44d348e4eec09d36006af6210fb7d4a3bb28457b2826cbb6ddcfa9e1d91a0"}
Jul 09 19:41:01.410716 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.410687    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zxkds" event={"ID":"389b1ece-cc85-4eb3-9111-a57d697c98c2","Type":"ContainerStarted","Data":"f78657ee672beceef059d3ed90994b74e1e704fe0b763a1bc80aa6e0c83f1a08"}
Jul 09 19:41:01.410920 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.410894    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:41:01.412759 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.412740    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v" event={"ID":"a05fdd56-7577-42c5-8345-1f4d1730331c","Type":"ContainerStarted","Data":"5606c428da1bfecc62519b9762da0c376c2c57a2de938ad36a209a428ec0ce1d"}
Jul 09 19:41:01.412847 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.412780    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v" event={"ID":"a05fdd56-7577-42c5-8345-1f4d1730331c","Type":"ContainerStarted","Data":"c07178398bc14ebda72baa767d0d2e6d68c3e8b4f2464ca62f65fd37bbe691c4"}
Jul 09 19:41:01.412847 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.412804    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v" event={"ID":"a05fdd56-7577-42c5-8345-1f4d1730331c","Type":"ContainerStarted","Data":"a9016cfa7addcad2db98c48abfc195f635457a1643e1b94f483fca97106e5922"}
Jul 09 19:41:01.440308 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.440249    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bjpjv" podStartSLOduration=67.094230255 podStartE2EDuration="1m12.440234198s" totalImagesPullingTime="5.346003943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:01.421945238 +0000 UTC m=+72.823080048" watchObservedRunningTime="2026-07-09 19:41:01.440234198 +0000 UTC m=+72.841369024"
Jul 09 19:41:01.440480 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.440340    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zxkds" podStartSLOduration=67.126978142 podStartE2EDuration="1m12.440331687s" totalImagesPullingTime="5.313353545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:01.43860579 +0000 UTC m=+72.839740614" watchObservedRunningTime="2026-07-09 19:41:01.440331687 +0000 UTC m=+72.841466499"
Jul 09 19:41:01.628276 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.628036    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 19:41:01.820839 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:01.820717    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 19:41:01.934153 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:01.934123    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3fc16d_d073_41d8_8f21_b07f370fea0c.slice/crio-686fb468b235bbd908f0d3248bd1dd15e6d3e5289d19359441239d76d8692bb1 WatchSource:0}: Error finding container 686fb468b235bbd908f0d3248bd1dd15e6d3e5289d19359441239d76d8692bb1: Status 404 returned error can't find the container with id 686fb468b235bbd908f0d3248bd1dd15e6d3e5289d19359441239d76d8692bb1
Jul 09 19:41:02.420017 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.419984    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k" event={"ID":"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad","Type":"ContainerStarted","Data":"46d9b10b9bba1c2f41a0a5bb4537c5d062ed8981deaab4b716ecd3b1951b131b"}
Jul 09 19:41:02.422225 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.421574    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg4ml" event={"ID":"1f31eedf-1c5a-4825-b852-3b66d081e2c1","Type":"ContainerStarted","Data":"459cb3d043e16625a6d81307329c126c915f18f5937bed5157d379c107caa0b8"}
Jul 09 19:41:02.424407 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.424375    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v" event={"ID":"a05fdd56-7577-42c5-8345-1f4d1730331c","Type":"ContainerStarted","Data":"aea5c816bf9bd78ee65d9dac8fad87bf4258b374a53b08b4454068ab326004c5"}
Jul 09 19:41:02.426590 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.426510    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerStarted","Data":"686fb468b235bbd908f0d3248bd1dd15e6d3e5289d19359441239d76d8692bb1"}
Jul 09 19:41:02.475945 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.475872    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9657v" podStartSLOduration=2.132579956 podStartE2EDuration="3.475847695s" totalImagesPullingTime="1.343267739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:02.475790186 +0000 UTC m=+73.876924996" watchObservedRunningTime="2026-07-09 19:41:02.475847695 +0000 UTC m=+73.876982506"
Jul 09 19:41:02.599837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.599793    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"]
Jul 09 19:41:02.605116 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.605093    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.608805 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.608773    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 09 19:41:02.608805 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.608785    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 09 19:41:02.609045 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.608969    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 09 19:41:02.609160 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.609145    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dbf0hs0o8si4n\""
Jul 09 19:41:02.609237 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.609186    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 09 19:41:02.609299 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.609267    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2wrh2\""
Jul 09 19:41:02.609402 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.609385    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 09 19:41:02.621768 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.621736    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"]
Jul 09 19:41:02.726164 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726124    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.726164 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726166    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztll8\" (UniqueName: \"kubernetes.io/projected/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-kube-api-access-ztll8\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.726680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726217    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-grpc-tls\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.726680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726298    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.726680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726327    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-metrics-client-ca\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.726680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726356    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.726680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726502    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-tls\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.726680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.726543    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.806020 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.805935    2527 patch_prober.go:28] interesting pod/image-registry-594f6499b-nz69n container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 19:41:02.806020 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.805988    2527 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-594f6499b-nz69n" podUID="ec51390a-ea19-4bda-a919-34fa8c5fbc8c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 19:41:02.827948 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.827911    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-tls\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.828096 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.827962    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.828096 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.827999    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.828096 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.828028    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztll8\" (UniqueName: \"kubernetes.io/projected/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-kube-api-access-ztll8\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.828096 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.828081    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-grpc-tls\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.828286 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.828154    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.828286 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.828178    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-metrics-client-ca\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.828286 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.828209    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.829468 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.829434    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-metrics-client-ca\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.831588 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.831323    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-tls\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.831588 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.831319    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.831588 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.831546    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.831966 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.831945    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.832529 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.832500    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-grpc-tls\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.832844 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.832824    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.836920 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.836900    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztll8\" (UniqueName: \"kubernetes.io/projected/8f863ade-b1d1-4b80-97f3-b1c54c607dcf-kube-api-access-ztll8\") pod \"thanos-querier-77cd7db9dc-w6wcj\" (UID: \"8f863ade-b1d1-4b80-97f3-b1c54c607dcf\") " pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:02.920699 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:02.920659    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:03.197963 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.197935    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"]
Jul 09 19:41:03.240554 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:03.240520    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f863ade_b1d1_4b80_97f3_b1c54c607dcf.slice/crio-c6516fd44108d94db90111dfef286b6410359a74beafa9b08b3fa3051e389bfa WatchSource:0}: Error finding container c6516fd44108d94db90111dfef286b6410359a74beafa9b08b3fa3051e389bfa: Status 404 returned error can't find the container with id c6516fd44108d94db90111dfef286b6410359a74beafa9b08b3fa3051e389bfa
Jul 09 19:41:03.437524 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.437431    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k" event={"ID":"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad","Type":"ContainerStarted","Data":"2a98cf1fa4e16d6f8103b6d8558cc3fa528280870adcfee409545e5446731cbc"}
Jul 09 19:41:03.437698 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.437537    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k" event={"ID":"a3fd7f1e-8d3e-4cf1-bc97-1975001adaad","Type":"ContainerStarted","Data":"69e47c9a3e468ec91e9fa90e219f539251abf88c06435ac9dfa3c5208d79f71c"}
Jul 09 19:41:03.438980 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.438954    2527 generic.go:356] "Generic (PLEG): container finished" podID="1f31eedf-1c5a-4825-b852-3b66d081e2c1" containerID="459cb3d043e16625a6d81307329c126c915f18f5937bed5157d379c107caa0b8" exitCode=0
Jul 09 19:41:03.439113 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.439015    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg4ml" event={"ID":"1f31eedf-1c5a-4825-b852-3b66d081e2c1","Type":"ContainerDied","Data":"459cb3d043e16625a6d81307329c126c915f18f5937bed5157d379c107caa0b8"}
Jul 09 19:41:03.440264 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.440241    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" event={"ID":"8f863ade-b1d1-4b80-97f3-b1c54c607dcf","Type":"ContainerStarted","Data":"c6516fd44108d94db90111dfef286b6410359a74beafa9b08b3fa3051e389bfa"}
Jul 09 19:41:03.441748 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.441726    2527 generic.go:356] "Generic (PLEG): container finished" podID="3f3fc16d-d073-41d8-8f21-b07f370fea0c" containerID="58d069d529f2474714f0ba37a4ffe776eb21ea6e8d0769f7b735e4c57462e4c4" exitCode=0
Jul 09 19:41:03.441828 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.441805    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerDied","Data":"58d069d529f2474714f0ba37a4ffe776eb21ea6e8d0769f7b735e4c57462e4c4"}
Jul 09 19:41:03.458341 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.458272    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-f5t7k" podStartSLOduration=2.931189331 podStartE2EDuration="4.458253308s" totalImagesPullingTime="1.527063977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:03.457215532 +0000 UTC m=+74.858350342" watchObservedRunningTime="2026-07-09 19:41:03.458253308 +0000 UTC m=+74.859388122"
Jul 09 19:41:03.999247 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:03.997695    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56889959ff-j7sh6"]
Jul 09 19:41:04.002351 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.001402    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.005494 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.005473    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-krnhf\""
Jul 09 19:41:04.005712 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.005688    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c0c37punhhqan\""
Jul 09 19:41:04.005826 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.005810    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 19:41:04.005964 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.005946    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 19:41:04.006045 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.006021    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 19:41:04.006240 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.006222    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 19:41:04.011938 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.011919    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56889959ff-j7sh6"]
Jul 09 19:41:04.141890 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.141847    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0c332ba1-106a-43d1-a7ad-77682b37f24c-audit-log\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.142051 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.141898    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-client-ca-bundle\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.142051 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.141954    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-secret-metrics-server-client-certs\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.142051 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.142019    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhm8l\" (UniqueName: \"kubernetes.io/projected/0c332ba1-106a-43d1-a7ad-77682b37f24c-kube-api-access-vhm8l\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.142178 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.142096    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-secret-metrics-server-tls\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.142178 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.142156    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c332ba1-106a-43d1-a7ad-77682b37f24c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.142268 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.142186    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0c332ba1-106a-43d1-a7ad-77682b37f24c-metrics-server-audit-profiles\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243165 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243127    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0c332ba1-106a-43d1-a7ad-77682b37f24c-audit-log\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243345 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243178    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-client-ca-bundle\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243345 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243253    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-secret-metrics-server-client-certs\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243345 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243296    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhm8l\" (UniqueName: \"kubernetes.io/projected/0c332ba1-106a-43d1-a7ad-77682b37f24c-kube-api-access-vhm8l\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243345 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243336    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-secret-metrics-server-tls\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243545 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243392    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c332ba1-106a-43d1-a7ad-77682b37f24c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243545 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243422    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0c332ba1-106a-43d1-a7ad-77682b37f24c-metrics-server-audit-profiles\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.243673 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.243586    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0c332ba1-106a-43d1-a7ad-77682b37f24c-audit-log\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.244563 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.244506    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c332ba1-106a-43d1-a7ad-77682b37f24c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.244685 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.244612    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0c332ba1-106a-43d1-a7ad-77682b37f24c-metrics-server-audit-profiles\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.246871 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.246821    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-secret-metrics-server-tls\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.247353 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.247331    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-client-ca-bundle\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.247915 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.247890    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0c332ba1-106a-43d1-a7ad-77682b37f24c-secret-metrics-server-client-certs\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.253152 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.253079    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhm8l\" (UniqueName: \"kubernetes.io/projected/0c332ba1-106a-43d1-a7ad-77682b37f24c-kube-api-access-vhm8l\") pod \"metrics-server-56889959ff-j7sh6\" (UID: \"0c332ba1-106a-43d1-a7ad-77682b37f24c\") " pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.315697 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.315639    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:04.329497 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.328579    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-594f6499b-nz69n"
Jul 09 19:41:04.349121 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.349092    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"]
Jul 09 19:41:04.354383 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.354355    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:04.357332 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.357306    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 09 19:41:04.357711 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.357545    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9szws\""
Jul 09 19:41:04.367498 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.367416    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"]
Jul 09 19:41:04.445991 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.445675    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqq2s\" (UniqueName: \"kubernetes.io/projected/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-kube-api-access-rqq2s\") pod \"monitoring-plugin-5f68df7f58-75sds\" (UID: \"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:04.445991 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.445803    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-75sds\" (UID: \"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:04.454299 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.454203    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg4ml" event={"ID":"1f31eedf-1c5a-4825-b852-3b66d081e2c1","Type":"ContainerStarted","Data":"1e60a68f6a6d34596eaac6be98a38ca67e77f94d4e154a8339b6ef445e97dcc1"}
Jul 09 19:41:04.454299 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.454256    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg4ml" event={"ID":"1f31eedf-1c5a-4825-b852-3b66d081e2c1","Type":"ContainerStarted","Data":"3b6dc64d531485ce59ae2683225242a133f6bda4682ac5ad16df9cde212f094e"}
Jul 09 19:41:04.476316 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.476232    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hg4ml" podStartSLOduration=3.6897226 podStartE2EDuration="5.476203118s" totalImagesPullingTime="1.685440354s" totalInitContainerRuntime="101.040164ms" isStatefulPod=true podCreationTimestamp="2026-07-09 19:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:04.473889195 +0000 UTC m=+75.875024007" watchObservedRunningTime="2026-07-09 19:41:04.476203118 +0000 UTC m=+75.877337922"
Jul 09 19:41:04.513073 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.512984    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56889959ff-j7sh6"]
Jul 09 19:41:04.517377 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:04.517350    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c332ba1_106a_43d1_a7ad_77682b37f24c.slice/crio-f9b66e7c7de267c3b846bd9bd09aa487a46ea8c5acafaa27697b8561d14d9c69 WatchSource:0}: Error finding container f9b66e7c7de267c3b846bd9bd09aa487a46ea8c5acafaa27697b8561d14d9c69: Status 404 returned error can't find the container with id f9b66e7c7de267c3b846bd9bd09aa487a46ea8c5acafaa27697b8561d14d9c69
Jul 09 19:41:04.547746 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.547201    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq2s\" (UniqueName: \"kubernetes.io/projected/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-kube-api-access-rqq2s\") pod \"monitoring-plugin-5f68df7f58-75sds\" (UID: \"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:04.547746 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.547263    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-75sds\" (UID: \"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:04.547746 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:04.547690    2527 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 09 19:41:04.548008 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:04.547759    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-monitoring-plugin-cert podName:86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39 nodeName:}" failed. No retries permitted until 2026-07-09 19:41:05.04773663 +0000 UTC m=+76.448871425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-75sds" (UID: "86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39") : secret "monitoring-plugin-cert" not found
Jul 09 19:41:04.556812 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.556778    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f979994b-qqvbg"]
Jul 09 19:41:04.560264 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.560230    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqq2s\" (UniqueName: \"kubernetes.io/projected/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-kube-api-access-rqq2s\") pod \"monitoring-plugin-5f68df7f58-75sds\" (UID: \"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:04.590396 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.590296    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54cd7f7657-445kh"]
Jul 09 19:41:04.597879 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.597848    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.606459 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.606427    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54cd7f7657-445kh"]
Jul 09 19:41:04.749299 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.749266    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-serving-cert\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.749299 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.749306    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-service-ca\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.749534 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.749475    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-oauth-serving-cert\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.749534 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.749518    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-trusted-ca-bundle\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.749634 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.749545    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-config\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.749634 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.749595    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw7qd\" (UniqueName: \"kubernetes.io/projected/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-kube-api-access-sw7qd\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.749634 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.749620    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-oauth-config\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.822059 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.822019    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-64cb78f845-h46ld"]
Jul 09 19:41:04.825747 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.825725    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.828544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.828518    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 19:41:04.828678 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.828589    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 19:41:04.828849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.828831    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 19:41:04.828977 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.828961    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 19:41:04.829285 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.829270    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hj9v8\""
Jul 09 19:41:04.829522 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.829507    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 19:41:04.834716 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.834695    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 19:41:04.839871 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.839847    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64cb78f845-h46ld"]
Jul 09 19:41:04.850663 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.850624    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-oauth-serving-cert\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.850779 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.850705    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-trusted-ca-bundle\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.850779 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.850748    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-config\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.850779 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.850775    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw7qd\" (UniqueName: \"kubernetes.io/projected/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-kube-api-access-sw7qd\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.850933 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.850807    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-oauth-config\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.850933 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.850877    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-serving-cert\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.850933 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.850906    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-service-ca\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.851384 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.851363    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-config\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.851468 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.851419    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-oauth-serving-cert\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.851570 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.851545    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-trusted-ca-bundle\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.851717 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.851595    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-service-ca\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.854079 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.854057    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-oauth-config\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.854222 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.854205    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-serving-cert\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.859984 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.859963    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw7qd\" (UniqueName: \"kubernetes.io/projected/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-kube-api-access-sw7qd\") pod \"console-54cd7f7657-445kh\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") " pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.909761 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.909723    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:04.951613 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951581    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkdk6\" (UniqueName: \"kubernetes.io/projected/a8658510-1390-4a81-8746-a441d5a0288e-kube-api-access-mkdk6\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.951811 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951677    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-federate-client-tls\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.951811 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951697    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-secret-telemeter-client\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.951811 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951761    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-serving-certs-ca-bundle\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.951811 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951792    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.952016 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951819    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-metrics-client-ca\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.952016 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951913    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-telemeter-client-tls\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:04.952016 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:04.951945    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053432 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053389    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053518    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkdk6\" (UniqueName: \"kubernetes.io/projected/a8658510-1390-4a81-8746-a441d5a0288e-kube-api-access-mkdk6\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053571    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-75sds\" (UID: \"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053691    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-federate-client-tls\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053733    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-secret-telemeter-client\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053780    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-serving-certs-ca-bundle\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053808    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053839    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-metrics-client-ca\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.053893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.053872    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-telemeter-client-tls\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.054473 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.054352    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.054768 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.054628    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-serving-certs-ca-bundle\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.054915 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.054872    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8658510-1390-4a81-8746-a441d5a0288e-metrics-client-ca\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.056695 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.056666    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-75sds\" (UID: \"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:05.057163 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.057135    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-federate-client-tls\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.057247 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.057195    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-telemeter-client-tls\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.058280 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.058257    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-secret-telemeter-client\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.058431 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.058410    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8658510-1390-4a81-8746-a441d5a0288e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.062581 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.062563    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkdk6\" (UniqueName: \"kubernetes.io/projected/a8658510-1390-4a81-8746-a441d5a0288e-kube-api-access-mkdk6\") pod \"telemeter-client-64cb78f845-h46ld\" (UID: \"a8658510-1390-4a81-8746-a441d5a0288e\") " pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.137196 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.137114    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld"
Jul 09 19:41:05.268175 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.268138    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:05.464724 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.463694    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6" event={"ID":"0c332ba1-106a-43d1-a7ad-77682b37f24c","Type":"ContainerStarted","Data":"f9b66e7c7de267c3b846bd9bd09aa487a46ea8c5acafaa27697b8561d14d9c69"}
Jul 09 19:41:05.771553 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.771251    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54cd7f7657-445kh"]
Jul 09 19:41:05.779126 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:05.779083    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf6b18a_694c_4da9_892d_a42cc95b4fcf.slice/crio-a4618a864273a2810af576d4f7e35fa71c48f74bcf99e6544782a6f88de2f832 WatchSource:0}: Error finding container a4618a864273a2810af576d4f7e35fa71c48f74bcf99e6544782a6f88de2f832: Status 404 returned error can't find the container with id a4618a864273a2810af576d4f7e35fa71c48f74bcf99e6544782a6f88de2f832
Jul 09 19:41:05.789568 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.789462    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"]
Jul 09 19:41:05.799517 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:05.799492    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64cb78f845-h46ld"]
Jul 09 19:41:05.806627 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:05.806116    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8658510_1390_4a81_8746_a441d5a0288e.slice/crio-08099e6338c93b9d2f15d47fb94957e92321b6bbf790ad15ee1b22a7377890a3 WatchSource:0}: Error finding container 08099e6338c93b9d2f15d47fb94957e92321b6bbf790ad15ee1b22a7377890a3: Status 404 returned error can't find the container with id 08099e6338c93b9d2f15d47fb94957e92321b6bbf790ad15ee1b22a7377890a3
Jul 09 19:41:06.027702 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.027608    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 19:41:06.033514 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.032023    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.036482 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.036454    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 19:41:06.036724 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.036707    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 19:41:06.037085 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.036826    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 19:41:06.037085 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.036935    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 19:41:06.037085 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.036953    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 19:41:06.037085 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037048    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 19:41:06.037367 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037100    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 19:41:06.037367 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037151    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ecavv13p79kp8\""
Jul 09 19:41:06.037367 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037352    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 19:41:06.037525 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037486    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mm6jb\""
Jul 09 19:41:06.039433 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037722    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 19:41:06.039433 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037863    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 19:41:06.039433 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.037988    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 19:41:06.039433 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.038045    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 19:41:06.041242 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.041092    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 19:41:06.045131 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.045107    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 19:41:06.057489 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.057465    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 19:41:06.171131 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171096    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7px9x\" (UniqueName: \"kubernetes.io/projected/596d64de-45da-47c0-af1f-e59dd020544a-kube-api-access-7px9x\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171206    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/596d64de-45da-47c0-af1f-e59dd020544a-config-out\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171242    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171271    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171287    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171319    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171574 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171368    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171574 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171413    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171574 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171434    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-web-config\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171574 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171450    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171574 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171477    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171574 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171529    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-config\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171608    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171625    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171666    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171686    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171727    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171760    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/596d64de-45da-47c0-af1f-e59dd020544a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171798    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.171876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.171856    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.274913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.274870    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.274913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.274913    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275145 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.274936    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275145 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.274956    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275145 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.274985    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275145 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275029    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/596d64de-45da-47c0-af1f-e59dd020544a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275145 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275078    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275145 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275119    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275160    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7px9x\" (UniqueName: \"kubernetes.io/projected/596d64de-45da-47c0-af1f-e59dd020544a-kube-api-access-7px9x\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275215    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/596d64de-45da-47c0-af1f-e59dd020544a-config-out\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275242    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275272    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275299    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275320    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275351    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275378    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275400    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-web-config\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275420    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275950 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275454    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275950 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275481    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-config\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275950 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275700    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.275950 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.275769    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.276162 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.276031    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.277108 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.277080    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.278499 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.278383    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.278607 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.278513    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/596d64de-45da-47c0-af1f-e59dd020544a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.279443 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.279118    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.279443 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.279162    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.279443 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.279236    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.280168 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.279499    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/596d64de-45da-47c0-af1f-e59dd020544a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.280168 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.279811    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/596d64de-45da-47c0-af1f-e59dd020544a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.280913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.280807    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.280913 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.280862    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/596d64de-45da-47c0-af1f-e59dd020544a-config-out\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.281256 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.280947    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-config\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.281371 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.281348    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.282610 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.282587    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-web-config\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.283126 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.283107    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.283512 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.283465    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.284072 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.284020    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/596d64de-45da-47c0-af1f-e59dd020544a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.285172 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.285151    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7px9x\" (UniqueName: \"kubernetes.io/projected/596d64de-45da-47c0-af1f-e59dd020544a-kube-api-access-7px9x\") pod \"prometheus-k8s-0\" (UID: \"596d64de-45da-47c0-af1f-e59dd020544a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.348472 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.348431    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:06.482668 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.481579    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:41:06.482668 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.481720    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:41:06.482884 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.482755    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:41:06.484874 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.484691    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" event={"ID":"8f863ade-b1d1-4b80-97f3-b1c54c607dcf","Type":"ContainerStarted","Data":"4358976c98eb2a16a697482001c2658716292a01b926e72817430c1658ca609b"}
Jul 09 19:41:06.484874 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.484779    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" event={"ID":"8f863ade-b1d1-4b80-97f3-b1c54c607dcf","Type":"ContainerStarted","Data":"be20f9386eb3bf36e68c6a1463f79dd101986760539e347473968dac520a4baf"}
Jul 09 19:41:06.484874 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.484824    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" event={"ID":"8f863ade-b1d1-4b80-97f3-b1c54c607dcf","Type":"ContainerStarted","Data":"a134de0ccb90105daa32ff3d35471f5e7a5cdfab4d40f2f725702535c2f54f64"}
Jul 09 19:41:06.487035 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.486950    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmfjg\" (UID: \"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc\") " pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:41:06.492598 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.488156    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds" event={"ID":"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39","Type":"ContainerStarted","Data":"e4de2d0bdb76f794154b452a678eaea043bd78867a48bc9c97163e968460866f"}
Jul 09 19:41:06.494603 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.493729    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54cd7f7657-445kh" event={"ID":"9cf6b18a-694c-4da9-892d-a42cc95b4fcf","Type":"ContainerStarted","Data":"981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8"}
Jul 09 19:41:06.494603 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.493780    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54cd7f7657-445kh" event={"ID":"9cf6b18a-694c-4da9-892d-a42cc95b4fcf","Type":"ContainerStarted","Data":"a4618a864273a2810af576d4f7e35fa71c48f74bcf99e6544782a6f88de2f832"}
Jul 09 19:41:06.496600 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.496522    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld" event={"ID":"a8658510-1390-4a81-8746-a441d5a0288e","Type":"ContainerStarted","Data":"08099e6338c93b9d2f15d47fb94957e92321b6bbf790ad15ee1b22a7377890a3"}
Jul 09 19:41:06.503541 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.503288    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerStarted","Data":"316ecd71d7f638c234698df820a107fe79d8aec868608be1f09e506cb0c30aa5"}
Jul 09 19:41:06.503541 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.503373    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerStarted","Data":"7ca9682925deaeab91bb38267b466601ff46612f8f4644bbcc0d06299264673b"}
Jul 09 19:41:06.503541 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.503419    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerStarted","Data":"ef72866d3e3c6093c6f236aa769658a58565a70e92417bb7ddc16753938aaaed"}
Jul 09 19:41:06.503541 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.503460    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerStarted","Data":"8e3a92fb181771cb0dd8a7ebddf9a51656af00674d4380c9365efd32b5740752"}
Jul 09 19:41:06.503541 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.503498    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerStarted","Data":"ccd641d408f296fe8c49577111471cbd41da31ee49326c823ef144c7222b1535"}
Jul 09 19:41:06.514960 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.514191    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54cd7f7657-445kh" podStartSLOduration=2.51417218 podStartE2EDuration="2.51417218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:41:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:06.512000981 +0000 UTC m=+77.913135793" watchObservedRunningTime="2026-07-09 19:41:06.51417218 +0000 UTC m=+77.915306972"
Jul 09 19:41:06.560447 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.558986    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 19:41:06.669819 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:06.669776    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vmfjg"
Jul 09 19:41:07.082559 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:07.082523    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596d64de_45da_47c0_af1f_e59dd020544a.slice/crio-7609a424789727b1f4c1319faf763a392b2387b5f041591dd6f925edd39dfc45 WatchSource:0}: Error finding container 7609a424789727b1f4c1319faf763a392b2387b5f041591dd6f925edd39dfc45: Status 404 returned error can't find the container with id 7609a424789727b1f4c1319faf763a392b2387b5f041591dd6f925edd39dfc45
Jul 09 19:41:07.514676 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:07.514570    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerStarted","Data":"7609a424789727b1f4c1319faf763a392b2387b5f041591dd6f925edd39dfc45"}
Jul 09 19:41:08.070856 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.070831    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:41:08.074876 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.074855    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:41:08.270068 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.269780    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vmfjg"]
Jul 09 19:41:08.277389 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:41:08.277025    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa4397f_ec7d_4c32_aab1_f79e9e4d3dbc.slice/crio-ee2ba9e1df5627839b707c40dd412b9dd6de22a7d66ad597e97ea1aaf349f9a2 WatchSource:0}: Error finding container ee2ba9e1df5627839b707c40dd412b9dd6de22a7d66ad597e97ea1aaf349f9a2: Status 404 returned error can't find the container with id ee2ba9e1df5627839b707c40dd412b9dd6de22a7d66ad597e97ea1aaf349f9a2
Jul 09 19:41:08.523470 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.523430    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" event={"ID":"8f863ade-b1d1-4b80-97f3-b1c54c607dcf","Type":"ContainerStarted","Data":"254d986b78b1a21d9e7fa2058f264dcc0a663d46307737edbbe6f7322eccaff5"}
Jul 09 19:41:08.523659 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.523517    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" event={"ID":"8f863ade-b1d1-4b80-97f3-b1c54c607dcf","Type":"ContainerStarted","Data":"8993cbfd3e456757ce756a381d5e5785c6b256ecb21de84db93f19a09b08805b"}
Jul 09 19:41:08.523659 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.523557    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" event={"ID":"8f863ade-b1d1-4b80-97f3-b1c54c607dcf","Type":"ContainerStarted","Data":"b3fa5c540bc7fb7bd707ae2c2cd59aa05dcb405fb1f55bd9963127ab08e85bca"}
Jul 09 19:41:08.523659 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.523597    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:08.525162 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.525133    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds" event={"ID":"86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39","Type":"ContainerStarted","Data":"e1b9d39600726a6de7416c323c08f414786e7f0d29519f6578ea7cdd1a41f9dd"}
Jul 09 19:41:08.525281 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.525257    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:08.527421 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.526979    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6" event={"ID":"0c332ba1-106a-43d1-a7ad-77682b37f24c","Type":"ContainerStarted","Data":"a8ea8d0e67d1fbded0be14643c5b1430a5a7f8cd2c4d37af78676d258abdc67e"}
Jul 09 19:41:08.528735 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.528714    2527 generic.go:356] "Generic (PLEG): container finished" podID="596d64de-45da-47c0-af1f-e59dd020544a" containerID="1e053767a62f7384bb44f3d8b6f4f01e7c6992dbef2e1796d38c31ae5833e67f" exitCode=0
Jul 09 19:41:08.528856 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.528740    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerDied","Data":"1e053767a62f7384bb44f3d8b6f4f01e7c6992dbef2e1796d38c31ae5833e67f"}
Jul 09 19:41:08.530254 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.530233    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmfjg" event={"ID":"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc","Type":"ContainerStarted","Data":"ee2ba9e1df5627839b707c40dd412b9dd6de22a7d66ad597e97ea1aaf349f9a2"}
Jul 09 19:41:08.531521 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.531505    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds"
Jul 09 19:41:08.532710 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.532685    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld" event={"ID":"a8658510-1390-4a81-8746-a441d5a0288e","Type":"ContainerStarted","Data":"1162c452bd1dc4de9e823b265dcac0e8a711ee47fc54598bccbc12b16a292c34"}
Jul 09 19:41:08.532801 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.532747    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld" event={"ID":"a8658510-1390-4a81-8746-a441d5a0288e","Type":"ContainerStarted","Data":"f0140fd6af032323b49e75e49ad96414d092450421361c118efb1587d38115f6"}
Jul 09 19:41:08.532801 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.532777    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld" event={"ID":"a8658510-1390-4a81-8746-a441d5a0288e","Type":"ContainerStarted","Data":"16477809c8cc675fa95b0ef794505c66f816736efc6e2b4735fa29797238af7b"}
Jul 09 19:41:08.536287 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.536266    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3f3fc16d-d073-41d8-8f21-b07f370fea0c","Type":"ContainerStarted","Data":"7f9064f845a9fed5307385db4d1cb01956172418c862813a0028afd39c7fcf87"}
Jul 09 19:41:08.548145 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.548082    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj" podStartSLOduration=2.017457899 podStartE2EDuration="6.54806383s" totalImagesPullingTime="4.530605931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:41:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:08.546158405 +0000 UTC m=+79.947293226" watchObservedRunningTime="2026-07-09 19:41:08.54806383 +0000 UTC m=+79.949198640"
Jul 09 19:41:08.598864 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.598754    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-64cb78f845-h46ld" podStartSLOduration=2.280695711 podStartE2EDuration="4.598733064s" totalImagesPullingTime="2.318037353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:08.596783101 +0000 UTC m=+79.997917914" watchObservedRunningTime="2026-07-09 19:41:08.598733064 +0000 UTC m=+79.999867876"
Jul 09 19:41:08.625724 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.625674    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.312585875 podStartE2EDuration="8.62562741s" totalImagesPullingTime="5.29992364s" totalInitContainerRuntime="13.117895ms" isStatefulPod=false podCreationTimestamp="2026-07-09 19:41:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:08.623906137 +0000 UTC m=+80.025040951" watchObservedRunningTime="2026-07-09 19:41:08.62562741 +0000 UTC m=+80.026762220"
Jul 09 19:41:08.645212 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.645158    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6" podStartSLOduration=2.046187868 podStartE2EDuration="5.645136116s" totalImagesPullingTime="3.598948248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:08.644405519 +0000 UTC m=+80.045540329" watchObservedRunningTime="2026-07-09 19:41:08.645136116 +0000 UTC m=+80.046270930"
Jul 09 19:41:08.664046 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:08.663985    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-75sds" podStartSLOduration=2.349210779 podStartE2EDuration="4.6639613s" totalImagesPullingTime="2.314750521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:08.663951172 +0000 UTC m=+80.065085983" watchObservedRunningTime="2026-07-09 19:41:08.6639613 +0000 UTC m=+80.065096107"
Jul 09 19:41:10.150544 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:10.150409    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54cd7f7657-445kh"]
Jul 09 19:41:14.557571 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.557541    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-77cd7db9dc-w6wcj"
Jul 09 19:41:14.578798 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.578758    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerStarted","Data":"74050ba78e38c487316a864709e8fa1c61e0d48cea8ade962a3c42f3963271f8"}
Jul 09 19:41:14.578983 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.578841    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerStarted","Data":"7a25d8f4639a7abfa63d9b09b887bfba2bb0c1ea0a912976e2fc7f55a654eead"}
Jul 09 19:41:14.578983 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.578882    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerStarted","Data":"8b6a8b580a18ecbd3ef72b26842a66404ce4946b13dad9690bf566407ac65ca7"}
Jul 09 19:41:14.578983 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.578919    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerStarted","Data":"34172857954ca7104da2f617bdff6f3e5e121647a07f22ca034fc2dde5375f2d"}
Jul 09 19:41:14.578983 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.578954    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerStarted","Data":"fc4da433cb6c358a018c1aa4a32b685ec6fb1ae5ca0eebc6e11a1b4ddc43a4bf"}
Jul 09 19:41:14.579151 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.578991    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"596d64de-45da-47c0-af1f-e59dd020544a","Type":"ContainerStarted","Data":"5c9a9db0b363f0a3d1c1f7efaaac303489f7acd01f56049b7a2c576586afbd89"}
Jul 09 19:41:14.580615 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.580575    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmfjg" event={"ID":"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc","Type":"ContainerStarted","Data":"7083f0da0958a21f03919d4a38fb4d823053dc832bb87d830987235264ace454"}
Jul 09 19:41:14.580800 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.580675    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmfjg" event={"ID":"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc","Type":"ContainerStarted","Data":"d1e07db507916883765db0e10ca34528b68a48cc3232eae79e9d10ad25ead863"}
Jul 09 19:41:14.610891 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.610830    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.382918904 podStartE2EDuration="8.610809161s" totalImagesPullingTime="5.192918576s" totalInitContainerRuntime="34.971681ms" isStatefulPod=false podCreationTimestamp="2026-07-09 19:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:14.608957193 +0000 UTC m=+86.010092004" watchObservedRunningTime="2026-07-09 19:41:14.610809161 +0000 UTC m=+86.011943972"
Jul 09 19:41:14.909951 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:14.909901    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:16.348904 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:16.348808    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:41:18.606451 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:18.606413    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmfjg" event={"ID":"4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc","Type":"ContainerStarted","Data":"c3d148a12650c70bdd8c83e83af05411cc4d9b35d365a4783911eebfa0a12348"}
Jul 09 19:41:18.625847 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:18.625796    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vmfjg" podStartSLOduration=19.409101488 podStartE2EDuration="28.625779046s" totalImagesPullingTime="9.216677558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:40:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:41:18.62474582 +0000 UTC m=+90.025880630" watchObservedRunningTime="2026-07-09 19:41:18.625779046 +0000 UTC m=+90.026913855"
Jul 09 19:41:20.952491 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:20.952460    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-968dc5c8f-lnln4"]
Jul 09 19:41:24.316712 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:24.316635    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:24.317338 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:24.316737    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:29.583807 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.583679    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67f979994b-qqvbg" podUID="fe56dd1c-2ce7-4082-8cfa-5055f692f40c" containerName="console" containerID="cri-o://97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0" gracePeriod=15
Jul 09 19:41:29.876099 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.876061    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:41:29.919036 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.918999    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-oauth-config\") pod \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") "
Jul 09 19:41:29.919198 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919077    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-oauth-serving-cert\") pod \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") "
Jul 09 19:41:29.919198 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919135    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2dt9\" (UniqueName: \"kubernetes.io/projected/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-kube-api-access-b2dt9\") pod \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") "
Jul 09 19:41:29.919198 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919189    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-service-ca\") pod \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") "
Jul 09 19:41:29.919376 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919213    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-config\") pod \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") "
Jul 09 19:41:29.919376 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919290    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-serving-cert\") pod \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\" (UID: \"fe56dd1c-2ce7-4082-8cfa-5055f692f40c\") "
Jul 09 19:41:29.919533 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919504    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fe56dd1c-2ce7-4082-8cfa-5055f692f40c" (UID: "fe56dd1c-2ce7-4082-8cfa-5055f692f40c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:29.919586 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919553    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-service-ca" (OuterVolumeSpecName: "service-ca") pod "fe56dd1c-2ce7-4082-8cfa-5055f692f40c" (UID: "fe56dd1c-2ce7-4082-8cfa-5055f692f40c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:29.919688 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.919630    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-config" (OuterVolumeSpecName: "console-config") pod "fe56dd1c-2ce7-4082-8cfa-5055f692f40c" (UID: "fe56dd1c-2ce7-4082-8cfa-5055f692f40c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:29.921545 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.921508    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fe56dd1c-2ce7-4082-8cfa-5055f692f40c" (UID: "fe56dd1c-2ce7-4082-8cfa-5055f692f40c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:41:29.921545 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.921533    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-kube-api-access-b2dt9" (OuterVolumeSpecName: "kube-api-access-b2dt9") pod "fe56dd1c-2ce7-4082-8cfa-5055f692f40c" (UID: "fe56dd1c-2ce7-4082-8cfa-5055f692f40c"). InnerVolumeSpecName "kube-api-access-b2dt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:41:29.921793 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:29.921565    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fe56dd1c-2ce7-4082-8cfa-5055f692f40c" (UID: "fe56dd1c-2ce7-4082-8cfa-5055f692f40c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:41:30.020998 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.020962    2527 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-oauth-config\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:30.020998 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.020993    2527 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-oauth-serving-cert\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:30.020998 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.021004    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2dt9\" (UniqueName: \"kubernetes.io/projected/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-kube-api-access-b2dt9\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:30.021228 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.021016    2527 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:30.021228 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.021024    2527 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-config\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:30.021228 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.021033    2527 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe56dd1c-2ce7-4082-8cfa-5055f692f40c-console-serving-cert\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:30.656455 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.656419    2527 generic.go:356] "Generic (PLEG): container finished" podID="fe56dd1c-2ce7-4082-8cfa-5055f692f40c" containerID="97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0" exitCode=0
Jul 09 19:41:30.656893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.656463    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f979994b-qqvbg"
Jul 09 19:41:30.656893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.656511    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f979994b-qqvbg" event={"ID":"fe56dd1c-2ce7-4082-8cfa-5055f692f40c","Type":"ContainerDied","Data":"97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0"}
Jul 09 19:41:30.656893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.656562    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f979994b-qqvbg" event={"ID":"fe56dd1c-2ce7-4082-8cfa-5055f692f40c","Type":"ContainerDied","Data":"5ca9548d684dd63a6f08a5e19ab50d4be52a1f3feab316e59631a2c5485b650e"}
Jul 09 19:41:30.656893 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.656580    2527 scope.go:117] "RemoveContainer" containerID="97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0"
Jul 09 19:41:30.665974 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.665955    2527 scope.go:117] "RemoveContainer" containerID="97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0"
Jul 09 19:41:30.666245 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:30.666222    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0\": container with ID starting with 97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0 not found: ID does not exist" containerID="97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0"
Jul 09 19:41:30.666308 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.666253    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0"} err="failed to get container status \"97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0\": rpc error: code = NotFound desc = could not find container \"97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0\": container with ID starting with 97be4eebd9d21c26120e7802f4d43637cf3d9460aaca7b184c3ce8c03fba54b0 not found: ID does not exist"
Jul 09 19:41:30.677911 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.677885    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f979994b-qqvbg"]
Jul 09 19:41:30.681269 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:30.681246    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67f979994b-qqvbg"]
Jul 09 19:41:31.077569 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:31.077532    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe56dd1c-2ce7-4082-8cfa-5055f692f40c" path="/var/lib/kubelet/pods/fe56dd1c-2ce7-4082-8cfa-5055f692f40c/volumes"
Jul 09 19:41:32.429454 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:32.429426    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zxkds"
Jul 09 19:41:35.177001 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.176958    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54cd7f7657-445kh" podUID="9cf6b18a-694c-4da9-892d-a42cc95b4fcf" containerName="console" containerID="cri-o://981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8" gracePeriod=15
Jul 09 19:41:35.455560 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.455524    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:35.570849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.570819    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-serving-cert\") pod \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") "
Jul 09 19:41:35.571031 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.570876    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-oauth-config\") pod \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") "
Jul 09 19:41:35.571031 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.570926    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-trusted-ca-bundle\") pod \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") "
Jul 09 19:41:35.571031 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.570946    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-oauth-serving-cert\") pod \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") "
Jul 09 19:41:35.571031 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.570974    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw7qd\" (UniqueName: \"kubernetes.io/projected/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-kube-api-access-sw7qd\") pod \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") "
Jul 09 19:41:35.571031 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.570999    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-service-ca\") pod \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") "
Jul 09 19:41:35.571031 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.571021    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-config\") pod \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\" (UID: \"9cf6b18a-694c-4da9-892d-a42cc95b4fcf\") "
Jul 09 19:41:35.571462 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.571406    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9cf6b18a-694c-4da9-892d-a42cc95b4fcf" (UID: "9cf6b18a-694c-4da9-892d-a42cc95b4fcf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:35.571462 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.571442    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9cf6b18a-694c-4da9-892d-a42cc95b4fcf" (UID: "9cf6b18a-694c-4da9-892d-a42cc95b4fcf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:35.571462 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.571446    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-service-ca" (OuterVolumeSpecName: "service-ca") pod "9cf6b18a-694c-4da9-892d-a42cc95b4fcf" (UID: "9cf6b18a-694c-4da9-892d-a42cc95b4fcf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:35.571613 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.571457    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-config" (OuterVolumeSpecName: "console-config") pod "9cf6b18a-694c-4da9-892d-a42cc95b4fcf" (UID: "9cf6b18a-694c-4da9-892d-a42cc95b4fcf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:35.573238 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.573206    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-kube-api-access-sw7qd" (OuterVolumeSpecName: "kube-api-access-sw7qd") pod "9cf6b18a-694c-4da9-892d-a42cc95b4fcf" (UID: "9cf6b18a-694c-4da9-892d-a42cc95b4fcf"). InnerVolumeSpecName "kube-api-access-sw7qd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:41:35.573338 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.573316    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9cf6b18a-694c-4da9-892d-a42cc95b4fcf" (UID: "9cf6b18a-694c-4da9-892d-a42cc95b4fcf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:41:35.573396 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.573353    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9cf6b18a-694c-4da9-892d-a42cc95b4fcf" (UID: "9cf6b18a-694c-4da9-892d-a42cc95b4fcf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:41:35.672614 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.672581    2527 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-trusted-ca-bundle\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:35.672614 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.672608    2527 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-oauth-serving-cert\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:35.672614 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.672617    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw7qd\" (UniqueName: \"kubernetes.io/projected/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-kube-api-access-sw7qd\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:35.672614 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.672627    2527 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:35.672901 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.672635    2527 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-config\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:35.672901 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.672687    2527 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-serving-cert\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:35.672901 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.672696    2527 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cf6b18a-694c-4da9-892d-a42cc95b4fcf-console-oauth-config\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:35.679780 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.679688    2527 generic.go:356] "Generic (PLEG): container finished" podID="9cf6b18a-694c-4da9-892d-a42cc95b4fcf" containerID="981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8" exitCode=0
Jul 09 19:41:35.679780 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.679746    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54cd7f7657-445kh"
Jul 09 19:41:35.679988 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.679788    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54cd7f7657-445kh" event={"ID":"9cf6b18a-694c-4da9-892d-a42cc95b4fcf","Type":"ContainerDied","Data":"981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8"}
Jul 09 19:41:35.679988 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.679853    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54cd7f7657-445kh" event={"ID":"9cf6b18a-694c-4da9-892d-a42cc95b4fcf","Type":"ContainerDied","Data":"a4618a864273a2810af576d4f7e35fa71c48f74bcf99e6544782a6f88de2f832"}
Jul 09 19:41:35.679988 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.679882    2527 scope.go:117] "RemoveContainer" containerID="981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8"
Jul 09 19:41:35.689123 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.689102    2527 scope.go:117] "RemoveContainer" containerID="981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8"
Jul 09 19:41:35.689381 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:35.689364    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8\": container with ID starting with 981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8 not found: ID does not exist" containerID="981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8"
Jul 09 19:41:35.689444 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.689390    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8"} err="failed to get container status \"981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8\": rpc error: code = NotFound desc = could not find container \"981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8\": container with ID starting with 981eed616584b1eff649143fb20bb8487e122488e619006c1515faba3422ddc8 not found: ID does not exist"
Jul 09 19:41:35.701572 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.701551    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54cd7f7657-445kh"]
Jul 09 19:41:35.706092 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:35.706070    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54cd7f7657-445kh"]
Jul 09 19:41:37.076495 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:37.076465    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf6b18a-694c-4da9-892d-a42cc95b4fcf" path="/var/lib/kubelet/pods/9cf6b18a-694c-4da9-892d-a42cc95b4fcf/volumes"
Jul 09 19:41:44.321516 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:44.321480    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:44.325292 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:44.325263    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56889959ff-j7sh6"
Jul 09 19:41:45.983154 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:45.983097    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-968dc5c8f-lnln4" podUID="f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" containerName="console" containerID="cri-o://bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b" gracePeriod=15
Jul 09 19:41:46.284780 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.284745    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:41:46.366252 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366210    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-serving-cert\") pod \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") "
Jul 09 19:41:46.366414 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366296    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-service-ca\") pod \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") "
Jul 09 19:41:46.366414 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366398    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-trusted-ca-bundle\") pod \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") "
Jul 09 19:41:46.366499 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366433    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-config\") pod \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") "
Jul 09 19:41:46.366499 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366455    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-oauth-serving-cert\") pod \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") "
Jul 09 19:41:46.366598 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366510    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-oauth-config\") pod \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") "
Jul 09 19:41:46.366598 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366531    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v6jp\" (UniqueName: \"kubernetes.io/projected/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-kube-api-access-4v6jp\") pod \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\" (UID: \"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604\") "
Jul 09 19:41:46.366990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366794    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-service-ca" (OuterVolumeSpecName: "service-ca") pod "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" (UID: "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:46.366990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366833    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-config" (OuterVolumeSpecName: "console-config") pod "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" (UID: "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:46.366990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366869    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" (UID: "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:46.366990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366894    2527 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:46.366990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366910    2527 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-config\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:46.366990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.366924    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" (UID: "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:41:46.368878 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.368854    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" (UID: "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:41:46.368878 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.368863    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-kube-api-access-4v6jp" (OuterVolumeSpecName: "kube-api-access-4v6jp") pod "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" (UID: "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604"). InnerVolumeSpecName "kube-api-access-4v6jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:41:46.368999 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.368890    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" (UID: "f4a7fa4f-7d2d-4e42-8ba1-0989aad24604"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:41:46.468382 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.468345    2527 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-trusted-ca-bundle\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:46.468382 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.468375    2527 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-oauth-serving-cert\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:46.468382 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.468384    2527 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-oauth-config\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:46.468382 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.468392    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v6jp\" (UniqueName: \"kubernetes.io/projected/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-kube-api-access-4v6jp\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:46.468635 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.468401    2527 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604-console-serving-cert\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:41:46.732640 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.732608    2527 generic.go:356] "Generic (PLEG): container finished" podID="f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" containerID="bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b" exitCode=0
Jul 09 19:41:46.732849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.732665    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968dc5c8f-lnln4"
Jul 09 19:41:46.732849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.732701    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968dc5c8f-lnln4" event={"ID":"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604","Type":"ContainerDied","Data":"bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b"}
Jul 09 19:41:46.732849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.732762    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968dc5c8f-lnln4" event={"ID":"f4a7fa4f-7d2d-4e42-8ba1-0989aad24604","Type":"ContainerDied","Data":"e5fed104adf4ee682664416498c938564d6c849ca9f90665441b957e0cd5e001"}
Jul 09 19:41:46.732849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.732789    2527 scope.go:117] "RemoveContainer" containerID="bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b"
Jul 09 19:41:46.742203 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.742183    2527 scope.go:117] "RemoveContainer" containerID="bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b"
Jul 09 19:41:46.742470 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:41:46.742450    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b\": container with ID starting with bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b not found: ID does not exist" containerID="bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b"
Jul 09 19:41:46.742530 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.742479    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b"} err="failed to get container status \"bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b\": rpc error: code = NotFound desc = could not find container \"bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b\": container with ID starting with bf125c89ac4584032d2f23b8a8945f25e1db4acbd9b8cb87592e93bdeb3d609b not found: ID does not exist"
Jul 09 19:41:46.755459 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.755432    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-968dc5c8f-lnln4"]
Jul 09 19:41:46.758995 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:46.758977    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-968dc5c8f-lnln4"]
Jul 09 19:41:47.077699 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:41:47.077636    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" path="/var/lib/kubelet/pods/f4a7fa4f-7d2d-4e42-8ba1-0989aad24604/volumes"
Jul 09 19:42:06.349123 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:42:06.349089    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:42:06.368488 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:42:06.368464    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:42:06.827923 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:42:06.827894    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 19:43:36.732892 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.732850    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-58z45"]
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733180    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf6b18a-694c-4da9-892d-a42cc95b4fcf" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733190    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf6b18a-694c-4da9-892d-a42cc95b4fcf" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733199    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe56dd1c-2ce7-4082-8cfa-5055f692f40c" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733206    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe56dd1c-2ce7-4082-8cfa-5055f692f40c" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733225    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733231    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733285    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf6b18a-694c-4da9-892d-a42cc95b4fcf" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733294    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe56dd1c-2ce7-4082-8cfa-5055f692f40c" containerName="console"
Jul 09 19:43:36.733448 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.733306    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4a7fa4f-7d2d-4e42-8ba1-0989aad24604" containerName="console"
Jul 09 19:43:36.736444 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.736428    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:36.739013 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.738998    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 19:43:36.740064 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.740049    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 19:43:36.740110 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.740052    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-srn2z\""
Jul 09 19:43:36.745420 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.745399    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-58z45"]
Jul 09 19:43:36.864670 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.864610    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dea21d79-75b0-4220-89f2-dd1731eb64b5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-58z45\" (UID: \"dea21d79-75b0-4220-89f2-dd1731eb64b5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:36.864670 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.864672    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxks8\" (UniqueName: \"kubernetes.io/projected/dea21d79-75b0-4220-89f2-dd1731eb64b5-kube-api-access-nxks8\") pod \"cert-manager-webhook-6486ffd997-58z45\" (UID: \"dea21d79-75b0-4220-89f2-dd1731eb64b5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:36.966009 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.965976    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dea21d79-75b0-4220-89f2-dd1731eb64b5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-58z45\" (UID: \"dea21d79-75b0-4220-89f2-dd1731eb64b5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:36.966009 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.966007    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxks8\" (UniqueName: \"kubernetes.io/projected/dea21d79-75b0-4220-89f2-dd1731eb64b5-kube-api-access-nxks8\") pod \"cert-manager-webhook-6486ffd997-58z45\" (UID: \"dea21d79-75b0-4220-89f2-dd1731eb64b5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:36.973998 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.973964    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dea21d79-75b0-4220-89f2-dd1731eb64b5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-58z45\" (UID: \"dea21d79-75b0-4220-89f2-dd1731eb64b5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:36.974182 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:36.974161    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxks8\" (UniqueName: \"kubernetes.io/projected/dea21d79-75b0-4220-89f2-dd1731eb64b5-kube-api-access-nxks8\") pod \"cert-manager-webhook-6486ffd997-58z45\" (UID: \"dea21d79-75b0-4220-89f2-dd1731eb64b5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:37.063402 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:37.063351    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:37.198261 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:37.198235    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-58z45"]
Jul 09 19:43:37.200745 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:43:37.200717    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddea21d79_75b0_4220_89f2_dd1731eb64b5.slice/crio-5b902818bc1f8a7edb499f5aac6eb52b6d0a8c96e5fcec15b962da39f3be4ca6 WatchSource:0}: Error finding container 5b902818bc1f8a7edb499f5aac6eb52b6d0a8c96e5fcec15b962da39f3be4ca6: Status 404 returned error can't find the container with id 5b902818bc1f8a7edb499f5aac6eb52b6d0a8c96e5fcec15b962da39f3be4ca6
Jul 09 19:43:38.178873 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:38.178791    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-58z45" event={"ID":"dea21d79-75b0-4220-89f2-dd1731eb64b5","Type":"ContainerStarted","Data":"5b902818bc1f8a7edb499f5aac6eb52b6d0a8c96e5fcec15b962da39f3be4ca6"}
Jul 09 19:43:41.194559 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:41.194522    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-58z45" event={"ID":"dea21d79-75b0-4220-89f2-dd1731eb64b5","Type":"ContainerStarted","Data":"44c7e9c439499c3c3ced70e49460c1b657417495128b96798f7e01b5e41624e2"}
Jul 09 19:43:41.195050 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:41.194639    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:41.217991 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:41.217938    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-58z45" podStartSLOduration=1.920483996 podStartE2EDuration="5.217919972s" totalImagesPullingTime="3.297435976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:43:41.215721742 +0000 UTC m=+232.616856552" watchObservedRunningTime="2026-07-09 19:43:41.217919972 +0000 UTC m=+232.619054781"
Jul 09 19:43:47.200902 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:47.200869    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-58z45"
Jul 09 19:43:56.353970 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.353937    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mtbmd"]
Jul 09 19:43:56.357604 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.357580    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.360484 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.360462    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9lrz6\""
Jul 09 19:43:56.367143 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.367120    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mtbmd"]
Jul 09 19:43:56.449262 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.449228    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd319ebc-ceec-4a90-a946-89f88e773cb6-bound-sa-token\") pod \"cert-manager-55b4d954d5-mtbmd\" (UID: \"dd319ebc-ceec-4a90-a946-89f88e773cb6\") " pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.449460 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.449300    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjw8b\" (UniqueName: \"kubernetes.io/projected/dd319ebc-ceec-4a90-a946-89f88e773cb6-kube-api-access-xjw8b\") pod \"cert-manager-55b4d954d5-mtbmd\" (UID: \"dd319ebc-ceec-4a90-a946-89f88e773cb6\") " pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.550387 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.550350    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd319ebc-ceec-4a90-a946-89f88e773cb6-bound-sa-token\") pod \"cert-manager-55b4d954d5-mtbmd\" (UID: \"dd319ebc-ceec-4a90-a946-89f88e773cb6\") " pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.550531 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.550402    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjw8b\" (UniqueName: \"kubernetes.io/projected/dd319ebc-ceec-4a90-a946-89f88e773cb6-kube-api-access-xjw8b\") pod \"cert-manager-55b4d954d5-mtbmd\" (UID: \"dd319ebc-ceec-4a90-a946-89f88e773cb6\") " pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.558598 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.558559    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd319ebc-ceec-4a90-a946-89f88e773cb6-bound-sa-token\") pod \"cert-manager-55b4d954d5-mtbmd\" (UID: \"dd319ebc-ceec-4a90-a946-89f88e773cb6\") " pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.558802 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.558783    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjw8b\" (UniqueName: \"kubernetes.io/projected/dd319ebc-ceec-4a90-a946-89f88e773cb6-kube-api-access-xjw8b\") pod \"cert-manager-55b4d954d5-mtbmd\" (UID: \"dd319ebc-ceec-4a90-a946-89f88e773cb6\") " pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.667514 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.667420    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mtbmd"
Jul 09 19:43:56.799882 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:56.799840    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mtbmd"]
Jul 09 19:43:56.802255 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:43:56.802226    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd319ebc_ceec_4a90_a946_89f88e773cb6.slice/crio-81b660539e30e26a5704bd3568ea869631dfb6acb5a47639698d828f6c139e2a WatchSource:0}: Error finding container 81b660539e30e26a5704bd3568ea869631dfb6acb5a47639698d828f6c139e2a: Status 404 returned error can't find the container with id 81b660539e30e26a5704bd3568ea869631dfb6acb5a47639698d828f6c139e2a
Jul 09 19:43:57.259809 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:57.259773    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mtbmd" event={"ID":"dd319ebc-ceec-4a90-a946-89f88e773cb6","Type":"ContainerStarted","Data":"20dbbe7acb13dd02fba56ca7e580f7785b0e26aeda51c55f5a34998ee3a59a43"}
Jul 09 19:43:57.259809 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:57.259822    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mtbmd" event={"ID":"dd319ebc-ceec-4a90-a946-89f88e773cb6","Type":"ContainerStarted","Data":"81b660539e30e26a5704bd3568ea869631dfb6acb5a47639698d828f6c139e2a"}
Jul 09 19:43:57.276130 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:43:57.276082    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mtbmd" podStartSLOduration=1.27606376 podStartE2EDuration="1.27606376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:43:57.275200086 +0000 UTC m=+248.676334896" watchObservedRunningTime="2026-07-09 19:43:57.27606376 +0000 UTC m=+248.677198570"
Jul 09 19:44:15.853720 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.853688    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"]
Jul 09 19:44:15.857470 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.857454    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:15.860545 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.860525    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 19:44:15.861803 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.861785    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 19:44:15.861930 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.861820    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 19:44:15.861930 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.861831    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 19:44:15.861930 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.861820    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hdr2l\""
Jul 09 19:44:15.861930 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.861820    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 19:44:15.867152 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.867132    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"]
Jul 09 19:44:15.915990 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.915948    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnkrr\" (UniqueName: \"kubernetes.io/projected/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-kube-api-access-pnkrr\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:15.916191 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.916002    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-manager-config\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:15.916191 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.916087    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-metrics-cert\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:15.916191 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:15.916156    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-cert\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.017256 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.017218    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-cert\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.017454 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.017282    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnkrr\" (UniqueName: \"kubernetes.io/projected/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-kube-api-access-pnkrr\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.017454 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.017306    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-manager-config\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.017454 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.017340    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-metrics-cert\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.018134 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.018065    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-manager-config\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.019965 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.019936    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-metrics-cert\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.020063 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.019983    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-cert\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.036208 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.036178    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnkrr\" (UniqueName: \"kubernetes.io/projected/49efb049-1b9e-4ff8-a111-9cbe7d9e2033-kube-api-access-pnkrr\") pod \"lws-controller-manager-748698b87b-8qr9s\" (UID: \"49efb049-1b9e-4ff8-a111-9cbe7d9e2033\") " pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.167827 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.167738    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:16.300304 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.300072    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"]
Jul 09 19:44:16.302430 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:44:16.302406    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49efb049_1b9e_4ff8_a111_9cbe7d9e2033.slice/crio-f3b98a340bc22c28628cda33cd933b5590effec6b319080435b38361b06f4c0f WatchSource:0}: Error finding container f3b98a340bc22c28628cda33cd933b5590effec6b319080435b38361b06f4c0f: Status 404 returned error can't find the container with id f3b98a340bc22c28628cda33cd933b5590effec6b319080435b38361b06f4c0f
Jul 09 19:44:16.338389 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:16.338355    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s" event={"ID":"49efb049-1b9e-4ff8-a111-9cbe7d9e2033","Type":"ContainerStarted","Data":"f3b98a340bc22c28628cda33cd933b5590effec6b319080435b38361b06f4c0f"}
Jul 09 19:44:19.352185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:19.352097    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s" event={"ID":"49efb049-1b9e-4ff8-a111-9cbe7d9e2033","Type":"ContainerStarted","Data":"3e3300654de71ef37c277d785c138c75e096728d96706fb17cf9a945b5b211b2"}
Jul 09 19:44:19.352573 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:19.352232    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:19.371380 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:19.371330    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s" podStartSLOduration=1.588421058 podStartE2EDuration="4.371314913s" totalImagesPullingTime="2.782893855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:44:19.368800546 +0000 UTC m=+270.769935356" watchObservedRunningTime="2026-07-09 19:44:19.371314913 +0000 UTC m=+270.772449722"
Jul 09 19:44:24.614088 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.614051    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"]
Jul 09 19:44:24.617705 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.617685    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.620680 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.620638    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 19:44:24.621268 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.621248    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kw7nt\""
Jul 09 19:44:24.621336 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.621272    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 19:44:24.621336 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.621308    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 19:44:24.622232 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.622211    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 19:44:24.636613 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.636587    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"]
Jul 09 19:44:24.698181 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.698143    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mqsj\" (UniqueName: \"kubernetes.io/projected/e92cab95-3930-4bef-b08a-d219ee3b3bd8-kube-api-access-9mqsj\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.698181 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.698188    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e92cab95-3930-4bef-b08a-d219ee3b3bd8-tmp\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.698413 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.698212    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e92cab95-3930-4bef-b08a-d219ee3b3bd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.698413 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.698287    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e92cab95-3930-4bef-b08a-d219ee3b3bd8-manifests\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.698413 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.698306    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e92cab95-3930-4bef-b08a-d219ee3b3bd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.799207 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.799167    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e92cab95-3930-4bef-b08a-d219ee3b3bd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.799391 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.799225    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mqsj\" (UniqueName: \"kubernetes.io/projected/e92cab95-3930-4bef-b08a-d219ee3b3bd8-kube-api-access-9mqsj\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.799391 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.799260    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e92cab95-3930-4bef-b08a-d219ee3b3bd8-tmp\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.799391 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.799286    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e92cab95-3930-4bef-b08a-d219ee3b3bd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.799391 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.799342    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e92cab95-3930-4bef-b08a-d219ee3b3bd8-manifests\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.799855 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.799830    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e92cab95-3930-4bef-b08a-d219ee3b3bd8-manifests\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.801706 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.801682    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e92cab95-3930-4bef-b08a-d219ee3b3bd8-tmp\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.801815 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.801760    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e92cab95-3930-4bef-b08a-d219ee3b3bd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.801985 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.801967    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e92cab95-3930-4bef-b08a-d219ee3b3bd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.808210 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.808190    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mqsj\" (UniqueName: \"kubernetes.io/projected/e92cab95-3930-4bef-b08a-d219ee3b3bd8-kube-api-access-9mqsj\") pod \"opendatahub-operator-controller-manager-776898495d-wg5wh\" (UID: \"e92cab95-3930-4bef-b08a-d219ee3b3bd8\") " pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:24.928397 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:24.928304    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:25.074968 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:44:25.074938    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92cab95_3930_4bef_b08a_d219ee3b3bd8.slice/crio-a084ebc9ab415f16dcecb547d512a16de26914caf955b12847f942971b184940 WatchSource:0}: Error finding container a084ebc9ab415f16dcecb547d512a16de26914caf955b12847f942971b184940: Status 404 returned error can't find the container with id a084ebc9ab415f16dcecb547d512a16de26914caf955b12847f942971b184940
Jul 09 19:44:25.078388 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:25.078362    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"]
Jul 09 19:44:25.384005 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:25.383963    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh" event={"ID":"e92cab95-3930-4bef-b08a-d219ee3b3bd8","Type":"ContainerStarted","Data":"a084ebc9ab415f16dcecb547d512a16de26914caf955b12847f942971b184940"}
Jul 09 19:44:28.400476 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:28.400434    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh" event={"ID":"e92cab95-3930-4bef-b08a-d219ee3b3bd8","Type":"ContainerStarted","Data":"07750dc825d6f708f3f505518cc3773bb315724325360774e8f9dafdd00037e6"}
Jul 09 19:44:29.406488 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:29.406454    2527 generic.go:356] "Generic (PLEG): container finished" podID="e92cab95-3930-4bef-b08a-d219ee3b3bd8" containerID="07750dc825d6f708f3f505518cc3773bb315724325360774e8f9dafdd00037e6" exitCode=0
Jul 09 19:44:29.406861 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:29.406538    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh" event={"ID":"e92cab95-3930-4bef-b08a-d219ee3b3bd8","Type":"ContainerDied","Data":"07750dc825d6f708f3f505518cc3773bb315724325360774e8f9dafdd00037e6"}
Jul 09 19:44:30.358713 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:30.358686    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-748698b87b-8qr9s"
Jul 09 19:44:30.413359 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:30.413326    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh" event={"ID":"e92cab95-3930-4bef-b08a-d219ee3b3bd8","Type":"ContainerStarted","Data":"e7fce2affb11a0c2fb5bfe7e755858179e73a01530394b3e709603e31719ddc4"}
Jul 09 19:44:30.413860 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:30.413470    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:30.473189 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:30.473139    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh" podStartSLOduration=1.8539426030000001 podStartE2EDuration="6.473120254s" totalImagesPullingTime="3.561917862s" totalInitContainerRuntime="1.057259789s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:44:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:44:30.472230602 +0000 UTC m=+281.873365412" watchObservedRunningTime="2026-07-09 19:44:30.473120254 +0000 UTC m=+281.874255064"
Jul 09 19:44:41.421203 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:41.421172    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-776898495d-wg5wh"
Jul 09 19:44:44.469622 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.469585    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"]
Jul 09 19:44:44.479744 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.479714    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.482863 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.482843    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 19:44:44.485573 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.485554    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qhgwl\""
Jul 09 19:44:44.487209 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.487184    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 19:44:44.487317 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.487196    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 19:44:44.487317 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.487197    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 19:44:44.503857 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.503827    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"]
Jul 09 19:44:44.587143 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.587108    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl8wb\" (UniqueName: \"kubernetes.io/projected/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-kube-api-access-hl8wb\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.587323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.587239    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-tls-certs\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.587323 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.587281    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-tmp\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.688616 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.688566    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl8wb\" (UniqueName: \"kubernetes.io/projected/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-kube-api-access-hl8wb\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.688823 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.688695    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-tls-certs\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.688823 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.688743    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-tmp\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.691857 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.691827    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-tmp\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.691978 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.691867    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-tls-certs\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.699950 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.699922    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl8wb\" (UniqueName: \"kubernetes.io/projected/1dfb2417-24c2-47b0-bcdb-d4394d3356ca-kube-api-access-hl8wb\") pod \"kube-auth-proxy-d5f4bc74d-rx7kq\" (UID: \"1dfb2417-24c2-47b0-bcdb-d4394d3356ca\") " pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.790459 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.790421    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"
Jul 09 19:44:44.940040 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:44.940016    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq"]
Jul 09 19:44:44.942402 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:44:44.942372    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfb2417_24c2_47b0_bcdb_d4394d3356ca.slice/crio-26043533220e4a3dc81d267160cf821f9a0e96910aba5d9c884b7f6090ac7e8e WatchSource:0}: Error finding container 26043533220e4a3dc81d267160cf821f9a0e96910aba5d9c884b7f6090ac7e8e: Status 404 returned error can't find the container with id 26043533220e4a3dc81d267160cf821f9a0e96910aba5d9c884b7f6090ac7e8e
Jul 09 19:44:45.485066 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:45.485022    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq" event={"ID":"1dfb2417-24c2-47b0-bcdb-d4394d3356ca","Type":"ContainerStarted","Data":"26043533220e4a3dc81d267160cf821f9a0e96910aba5d9c884b7f6090ac7e8e"}
Jul 09 19:44:48.502939 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:48.502849    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq" event={"ID":"1dfb2417-24c2-47b0-bcdb-d4394d3356ca","Type":"ContainerStarted","Data":"fcc6485c9e84c08929eeec82cb4404cfc265b958de32cc6942d71d1704cc1878"}
Jul 09 19:44:48.524714 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:48.524620    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d5f4bc74d-rx7kq" podStartSLOduration=1.339320675 podStartE2EDuration="4.524597505s" totalImagesPullingTime="3.18527683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:44:48.522532866 +0000 UTC m=+299.923667678" watchObservedRunningTime="2026-07-09 19:44:48.524597505 +0000 UTC m=+299.925732317"
Jul 09 19:44:48.954774 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:44:48.954747    2527 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 19:46:12.207557 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.207528    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78f856f565-t9j6z"]
Jul 09 19:46:12.210965 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.210948    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.216921 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.216890    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 19:46:12.217068 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.216941    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 19:46:12.217068 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.216969    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 19:46:12.217068 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.216989    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wztb4\""
Jul 09 19:46:12.217068 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.216941    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 19:46:12.217287 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.217217    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 19:46:12.217287 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.217251    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 19:46:12.217372 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.217293    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 19:46:12.225313 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.225294    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 19:46:12.228385 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.228363    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f856f565-t9j6z"]
Jul 09 19:46:12.303023 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.302988    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-trusted-ca-bundle\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.303194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.303042    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ggmn\" (UniqueName: \"kubernetes.io/projected/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-kube-api-access-7ggmn\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.303194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.303118    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-oauth-config\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.303194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.303182    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-serving-cert\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.303380 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.303211    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-config\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.303380 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.303276    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-service-ca\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.303380 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.303306    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-oauth-serving-cert\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.403894 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.403863    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ggmn\" (UniqueName: \"kubernetes.io/projected/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-kube-api-access-7ggmn\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.403919    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-oauth-config\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.403966    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-serving-cert\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.403990    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-config\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.404037    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-service-ca\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.404065    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-oauth-serving-cert\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404366 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.404111    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-trusted-ca-bundle\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404830 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.404805    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-service-ca\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.404961 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.404856    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-oauth-serving-cert\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.405178 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.405155    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-config\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.405567 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.405544    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-trusted-ca-bundle\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.406751 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.406730    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-serving-cert\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.406855 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.406795    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-console-oauth-config\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.413413 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.413386    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ggmn\" (UniqueName: \"kubernetes.io/projected/07ef7226-8d24-42c9-b16a-55fdd36dbbb3-kube-api-access-7ggmn\") pod \"console-78f856f565-t9j6z\" (UID: \"07ef7226-8d24-42c9-b16a-55fdd36dbbb3\") " pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.521000 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.520964    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:12.662199 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.662167    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78f856f565-t9j6z"]
Jul 09 19:46:12.664726 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:46:12.664697    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ef7226_8d24_42c9_b16a_55fdd36dbbb3.slice/crio-55023d313453d8d28bae5705cd990dd1d428dbd1841065a0f0ab8265f9655652 WatchSource:0}: Error finding container 55023d313453d8d28bae5705cd990dd1d428dbd1841065a0f0ab8265f9655652: Status 404 returned error can't find the container with id 55023d313453d8d28bae5705cd990dd1d428dbd1841065a0f0ab8265f9655652
Jul 09 19:46:12.893412 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.893323    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f856f565-t9j6z" event={"ID":"07ef7226-8d24-42c9-b16a-55fdd36dbbb3","Type":"ContainerStarted","Data":"e23c18f89646729279ebc13e4f9dabcf15a6e08aaf6f4dbde6fe9aabb0a11a9e"}
Jul 09 19:46:12.893412 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.893374    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78f856f565-t9j6z" event={"ID":"07ef7226-8d24-42c9-b16a-55fdd36dbbb3","Type":"ContainerStarted","Data":"55023d313453d8d28bae5705cd990dd1d428dbd1841065a0f0ab8265f9655652"}
Jul 09 19:46:12.917933 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:12.917868    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78f856f565-t9j6z" podStartSLOduration=0.917848242 podStartE2EDuration="917.848242ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:46:12.915628799 +0000 UTC m=+384.316763610" watchObservedRunningTime="2026-07-09 19:46:12.917848242 +0000 UTC m=+384.318983052"
Jul 09 19:46:22.521442 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:22.521408    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:22.521887 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:22.521709    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:22.525944 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:22.525922    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:22.941349 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:22.941267    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78f856f565-t9j6z"
Jul 09 19:46:48.367094 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.367055    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"]
Jul 09 19:46:48.370558 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.370539    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.373454 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.373433    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 19:46:48.373759 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.373725    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 19:46:48.373858 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.373758    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jvqwt\""
Jul 09 19:46:48.388130 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.388108    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"]
Jul 09 19:46:48.542733 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.542691    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.542923 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.542839    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5ztg\" (UniqueName: \"kubernetes.io/projected/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-kube-api-access-r5ztg\") pod \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.643773 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.643637    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5ztg\" (UniqueName: \"kubernetes.io/projected/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-kube-api-access-r5ztg\") pod \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.643773 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.643744    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.644122 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.644104    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.652563 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.652530    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5ztg\" (UniqueName: \"kubernetes.io/projected/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-kube-api-access-r5ztg\") pod \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.681502 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.681469    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:48.709868 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.709835    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"]
Jul 09 19:46:48.716284 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:48.716248    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"]
Jul 09 19:46:49.687861 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:46:49.687820    2527 log.go:32] "RunPodSandbox from runtime service failed" err=<
Jul 09 19:46:49.687861 ip-10-0-136-128 kubenswrapper[2527]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-jjbpd_kuadrant-system_0b6e06f8-aa27-4bf6-a50d-12e0c503bff0_0(cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-jjbpd to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d" Netns:"/var/run/netns/e52cc24a-9305-49db-a475-3ccea85ea900" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-jjbpd;K8S_POD_INFRA_CONTAINER_ID=cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d;K8S_POD_UID=0b6e06f8-aa27-4bf6-a50d-12e0c503bff0" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d network default NAD default NAD key ] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d network default NAD default NAD key ] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-55467d9f7-jjbpd" not found
Jul 09 19:46:49.687861 ip-10-0-136-128 kubenswrapper[2527]:         '
Jul 09 19:46:49.687861 ip-10-0-136-128 kubenswrapper[2527]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 09 19:46:49.687861 ip-10-0-136-128 kubenswrapper[2527]:  >
Jul 09 19:46:49.688237 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:46:49.687900    2527 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Jul 09 19:46:49.688237 ip-10-0-136-128 kubenswrapper[2527]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-jjbpd_kuadrant-system_0b6e06f8-aa27-4bf6-a50d-12e0c503bff0_0(cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-jjbpd to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d" Netns:"/var/run/netns/e52cc24a-9305-49db-a475-3ccea85ea900" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-jjbpd;K8S_POD_INFRA_CONTAINER_ID=cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d;K8S_POD_UID=0b6e06f8-aa27-4bf6-a50d-12e0c503bff0" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d network default NAD default NAD key ] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd cc8b60d3a9d2924c41b29e1abe1faa8bb8527f176ba2ad3d4876fd958eb2150d network default NAD default NAD key ] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-55467d9f7-jjbpd" not found
Jul 09 19:46:49.688237 ip-10-0-136-128 kubenswrapper[2527]:         '
Jul 09 19:46:49.688237 ip-10-0-136-128 kubenswrapper[2527]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 09 19:46:49.688237 ip-10-0-136-128 kubenswrapper[2527]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:50.064491 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.064458    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:50.072764 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.072736    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:50.075365 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:46:50.075340    2527 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" is forbidden: User \"system:node:ip-10-0-136-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-128.ec2.internal' and this object" podUID="0b6e06f8-aa27-4bf6-a50d-12e0c503bff0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:50.256916 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.256870    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5ztg\" (UniqueName: \"kubernetes.io/projected/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-kube-api-access-r5ztg\") pod \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") "
Jul 09 19:46:50.256916 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.256925    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-extensions-socket-volume\") pod \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\" (UID: \"0b6e06f8-aa27-4bf6-a50d-12e0c503bff0\") "
Jul 09 19:46:50.257219 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.257187    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b6e06f8-aa27-4bf6-a50d-12e0c503bff0" (UID: "0b6e06f8-aa27-4bf6-a50d-12e0c503bff0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:46:50.257338 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.257282    2527 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-extensions-socket-volume\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:46:50.259125 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.259099    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-kube-api-access-r5ztg" (OuterVolumeSpecName: "kube-api-access-r5ztg") pod "0b6e06f8-aa27-4bf6-a50d-12e0c503bff0" (UID: "0b6e06f8-aa27-4bf6-a50d-12e0c503bff0"). InnerVolumeSpecName "kube-api-access-r5ztg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:46:50.357925 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:50.357849    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5ztg\" (UniqueName: \"kubernetes.io/projected/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0-kube-api-access-r5ztg\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:46:51.072195 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:51.072167    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:51.074539 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:46:51.074502    2527 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jjbpd\" is forbidden: User \"system:node:ip-10-0-136-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-128.ec2.internal' and this object" podUID="0b6e06f8-aa27-4bf6-a50d-12e0c503bff0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jjbpd"
Jul 09 19:46:51.077861 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:46:51.077742    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b6e06f8-aa27-4bf6-a50d-12e0c503bff0" path="/var/lib/kubelet/pods/0b6e06f8-aa27-4bf6-a50d-12e0c503bff0/volumes"
Jul 09 19:47:06.435934 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.435896    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"]
Jul 09 19:47:06.439513 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.439493    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.445401 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.445376    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jvqwt\""
Jul 09 19:47:06.445729 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.445709    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 19:47:06.445793 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.445729    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 19:47:06.461036 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.461011    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"]
Jul 09 19:47:06.504438 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.504406    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djnnd\" (UniqueName: \"kubernetes.io/projected/e47d906f-6517-4b5c-a814-cabce5a146f1-kube-api-access-djnnd\") pod \"kuadrant-operator-controller-manager-86878c769f-nbrw4\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.504603 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.504461    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e47d906f-6517-4b5c-a814-cabce5a146f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nbrw4\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.605894 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.605853    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djnnd\" (UniqueName: \"kubernetes.io/projected/e47d906f-6517-4b5c-a814-cabce5a146f1-kube-api-access-djnnd\") pod \"kuadrant-operator-controller-manager-86878c769f-nbrw4\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.606058 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.605925    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e47d906f-6517-4b5c-a814-cabce5a146f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nbrw4\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.606346 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.606327    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e47d906f-6517-4b5c-a814-cabce5a146f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nbrw4\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.623320 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.623288    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djnnd\" (UniqueName: \"kubernetes.io/projected/e47d906f-6517-4b5c-a814-cabce5a146f1-kube-api-access-djnnd\") pod \"kuadrant-operator-controller-manager-86878c769f-nbrw4\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.749942 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.749855    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:06.888272 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.888238    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"]
Jul 09 19:47:06.891513 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:47:06.891479    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47d906f_6517_4b5c_a814_cabce5a146f1.slice/crio-48fc3aed2c725b244f95fde99f8c64a29ec4c0863d181c896fe21c9e269977d9 WatchSource:0}: Error finding container 48fc3aed2c725b244f95fde99f8c64a29ec4c0863d181c896fe21c9e269977d9: Status 404 returned error can't find the container with id 48fc3aed2c725b244f95fde99f8c64a29ec4c0863d181c896fe21c9e269977d9
Jul 09 19:47:06.893934 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:06.893919    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 19:47:07.151141 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:07.151104    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4" event={"ID":"e47d906f-6517-4b5c-a814-cabce5a146f1","Type":"ContainerStarted","Data":"48fc3aed2c725b244f95fde99f8c64a29ec4c0863d181c896fe21c9e269977d9"}
Jul 09 19:47:12.180326 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:12.180222    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4" event={"ID":"e47d906f-6517-4b5c-a814-cabce5a146f1","Type":"ContainerStarted","Data":"cb92e46cb050db345596a6a1169214345f35942d3a9b6f9ab3691834fe63ba7d"}
Jul 09 19:47:12.180326 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:12.180296    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:12.209951 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:12.209898    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4" podStartSLOduration=1.239931731 podStartE2EDuration="6.209875548s" totalImagesPullingTime="4.969943817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:47:12.206516021 +0000 UTC m=+443.607650830" watchObservedRunningTime="2026-07-09 19:47:12.209875548 +0000 UTC m=+443.611010358"
Jul 09 19:47:23.188304 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:23.188271    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:47:42.642441 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.642403    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-d8h6s"]
Jul 09 19:47:42.646085 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.646064    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d8h6s"
Jul 09 19:47:42.649937 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.649915    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9w5tp\""
Jul 09 19:47:42.656122 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.656099    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d8h6s"]
Jul 09 19:47:42.735111 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.735080    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcpxf\" (UniqueName: \"kubernetes.io/projected/7d66ef5a-bcdc-451d-a275-52e4a77bad43-kube-api-access-tcpxf\") pod \"authorino-75d556bd77-d8h6s\" (UID: \"7d66ef5a-bcdc-451d-a275-52e4a77bad43\") " pod="kuadrant-system/authorino-75d556bd77-d8h6s"
Jul 09 19:47:42.835590 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.835560    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpxf\" (UniqueName: \"kubernetes.io/projected/7d66ef5a-bcdc-451d-a275-52e4a77bad43-kube-api-access-tcpxf\") pod \"authorino-75d556bd77-d8h6s\" (UID: \"7d66ef5a-bcdc-451d-a275-52e4a77bad43\") " pod="kuadrant-system/authorino-75d556bd77-d8h6s"
Jul 09 19:47:42.844350 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.844318    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcpxf\" (UniqueName: \"kubernetes.io/projected/7d66ef5a-bcdc-451d-a275-52e4a77bad43-kube-api-access-tcpxf\") pod \"authorino-75d556bd77-d8h6s\" (UID: \"7d66ef5a-bcdc-451d-a275-52e4a77bad43\") " pod="kuadrant-system/authorino-75d556bd77-d8h6s"
Jul 09 19:47:42.956112 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:42.956020    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d8h6s"
Jul 09 19:47:43.087828 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:43.087807    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d8h6s"]
Jul 09 19:47:43.090268 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:47:43.090233    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d66ef5a_bcdc_451d_a275_52e4a77bad43.slice/crio-18466d1399d16abd1cea42d4c6a6f474f95c36816e05a9cc227ec7c2724963c9 WatchSource:0}: Error finding container 18466d1399d16abd1cea42d4c6a6f474f95c36816e05a9cc227ec7c2724963c9: Status 404 returned error can't find the container with id 18466d1399d16abd1cea42d4c6a6f474f95c36816e05a9cc227ec7c2724963c9
Jul 09 19:47:43.333837 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:43.333797    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d8h6s" event={"ID":"7d66ef5a-bcdc-451d-a275-52e4a77bad43","Type":"ContainerStarted","Data":"18466d1399d16abd1cea42d4c6a6f474f95c36816e05a9cc227ec7c2724963c9"}
Jul 09 19:47:46.352056 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:46.352023    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d8h6s" event={"ID":"7d66ef5a-bcdc-451d-a275-52e4a77bad43","Type":"ContainerStarted","Data":"a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3"}
Jul 09 19:47:46.370137 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:47:46.370086    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-d8h6s" podStartSLOduration=1.251098538 podStartE2EDuration="4.370071772s" totalImagesPullingTime="3.118973234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:47:46.368775496 +0000 UTC m=+477.769910309" watchObservedRunningTime="2026-07-09 19:47:46.370071772 +0000 UTC m=+477.771206581"
Jul 09 19:48:16.540589 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:16.540553    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d8h6s"]
Jul 09 19:48:16.541107 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:16.540838    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-d8h6s" podUID="7d66ef5a-bcdc-451d-a275-52e4a77bad43" containerName="authorino" containerID="cri-o://a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3" gracePeriod=30
Jul 09 19:48:16.796359 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:16.796297    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d8h6s"
Jul 09 19:48:16.948898 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:16.948866    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcpxf\" (UniqueName: \"kubernetes.io/projected/7d66ef5a-bcdc-451d-a275-52e4a77bad43-kube-api-access-tcpxf\") pod \"7d66ef5a-bcdc-451d-a275-52e4a77bad43\" (UID: \"7d66ef5a-bcdc-451d-a275-52e4a77bad43\") "
Jul 09 19:48:16.951185 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:16.951160    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d66ef5a-bcdc-451d-a275-52e4a77bad43-kube-api-access-tcpxf" (OuterVolumeSpecName: "kube-api-access-tcpxf") pod "7d66ef5a-bcdc-451d-a275-52e4a77bad43" (UID: "7d66ef5a-bcdc-451d-a275-52e4a77bad43"). InnerVolumeSpecName "kube-api-access-tcpxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:48:17.050348 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.050273    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcpxf\" (UniqueName: \"kubernetes.io/projected/7d66ef5a-bcdc-451d-a275-52e4a77bad43-kube-api-access-tcpxf\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:48:17.496637 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.496551    2527 generic.go:356] "Generic (PLEG): container finished" podID="7d66ef5a-bcdc-451d-a275-52e4a77bad43" containerID="a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3" exitCode=0
Jul 09 19:48:17.496637 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.496616    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d8h6s"
Jul 09 19:48:17.496849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.496621    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d8h6s" event={"ID":"7d66ef5a-bcdc-451d-a275-52e4a77bad43","Type":"ContainerDied","Data":"a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3"}
Jul 09 19:48:17.496849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.496702    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d8h6s" event={"ID":"7d66ef5a-bcdc-451d-a275-52e4a77bad43","Type":"ContainerDied","Data":"18466d1399d16abd1cea42d4c6a6f474f95c36816e05a9cc227ec7c2724963c9"}
Jul 09 19:48:17.496849 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.496720    2527 scope.go:117] "RemoveContainer" containerID="a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3"
Jul 09 19:48:17.507224 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.507203    2527 scope.go:117] "RemoveContainer" containerID="a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3"
Jul 09 19:48:17.507528 ip-10-0-136-128 kubenswrapper[2527]: E0709 19:48:17.507507    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3\": container with ID starting with a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3 not found: ID does not exist" containerID="a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3"
Jul 09 19:48:17.507601 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.507542    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3"} err="failed to get container status \"a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3\": rpc error: code = NotFound desc = could not find container \"a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3\": container with ID starting with a03fce3cdaec6a19f2c5f96a520716d691bff31939db18656db62fa1334ac4f3 not found: ID does not exist"
Jul 09 19:48:17.522200 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.522169    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d8h6s"]
Jul 09 19:48:17.524251 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:17.524232    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d8h6s"]
Jul 09 19:48:19.077843 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:48:19.077805    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d66ef5a-bcdc-451d-a275-52e4a77bad43" path="/var/lib/kubelet/pods/7d66ef5a-bcdc-451d-a275-52e4a77bad43/volumes"
Jul 09 19:49:22.002662 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.002611    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"]
Jul 09 19:49:22.003103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.003017    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d66ef5a-bcdc-451d-a275-52e4a77bad43" containerName="authorino"
Jul 09 19:49:22.003103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.003028    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d66ef5a-bcdc-451d-a275-52e4a77bad43" containerName="authorino"
Jul 09 19:49:22.003103 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.003083    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d66ef5a-bcdc-451d-a275-52e4a77bad43" containerName="authorino"
Jul 09 19:49:22.006209 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.006187    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.010382 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.010362    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 19:49:22.010382 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.010364    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mpnwx\""
Jul 09 19:49:22.010563 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.010399    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 09 19:49:22.010563 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.010403    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 19:49:22.019683 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.019629    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"]
Jul 09 19:49:22.055884 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.055848    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.055884 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.055882    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f28t9\" (UniqueName: \"kubernetes.io/projected/795a0d43-6157-4f80-a920-e660ed918d28-kube-api-access-f28t9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.056142 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.055912    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/795a0d43-6157-4f80-a920-e660ed918d28-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.056142 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.055933    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.056142 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.056006    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.056142 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.056073    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.156872 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.156836    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157074 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.156886    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157074 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.156963    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157074 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.156986    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f28t9\" (UniqueName: \"kubernetes.io/projected/795a0d43-6157-4f80-a920-e660ed918d28-kube-api-access-f28t9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157074 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.157027    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/795a0d43-6157-4f80-a920-e660ed918d28-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157074 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.157063    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157384 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.157352    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157496 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.157397    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.157496 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.157370    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.159623 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.159600    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/795a0d43-6157-4f80-a920-e660ed918d28-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.159780 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.159759    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/795a0d43-6157-4f80-a920-e660ed918d28-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.165880 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.165854    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f28t9\" (UniqueName: \"kubernetes.io/projected/795a0d43-6157-4f80-a920-e660ed918d28-kube-api-access-f28t9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx\" (UID: \"795a0d43-6157-4f80-a920-e660ed918d28\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.318283 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.318243    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:22.458013 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.457986    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"]
Jul 09 19:49:22.460595 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:49:22.460563    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795a0d43_6157_4f80_a920_e660ed918d28.slice/crio-998fd9fb302eae28e5c91995c30ae9295c06a410762408797b3903c16bad41cd WatchSource:0}: Error finding container 998fd9fb302eae28e5c91995c30ae9295c06a410762408797b3903c16bad41cd: Status 404 returned error can't find the container with id 998fd9fb302eae28e5c91995c30ae9295c06a410762408797b3903c16bad41cd
Jul 09 19:49:22.811562 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:22.811525    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx" event={"ID":"795a0d43-6157-4f80-a920-e660ed918d28","Type":"ContainerStarted","Data":"998fd9fb302eae28e5c91995c30ae9295c06a410762408797b3903c16bad41cd"}
Jul 09 19:49:23.818597 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:23.818560    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx" event={"ID":"795a0d43-6157-4f80-a920-e660ed918d28","Type":"ContainerStarted","Data":"5970628dbbc116f0a0b8c9e54f06c24fdbed8156475c1a8084794b73788fac09"}
Jul 09 19:49:23.841307 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:23.841247    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx" podStartSLOduration=1.6470950850000001 podStartE2EDuration="2.841230873s" totalImagesPullingTime="1.194135788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:49:23.837898406 +0000 UTC m=+575.239033216" watchObservedRunningTime="2026-07-09 19:49:23.841230873 +0000 UTC m=+575.242365682"
Jul 09 19:49:32.319069 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:32.319029    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:32.319454 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:32.319102    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:32.331536 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:32.331508    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:32.872508 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:32.872480    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx"
Jul 09 19:49:39.099222 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.099186    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"]
Jul 09 19:49:39.103018 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.102998    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.105748 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.105729    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 09 19:49:39.123563 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.123533    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"]
Jul 09 19:49:39.230358 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.230270    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.230358 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.230309    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47kbj\" (UniqueName: \"kubernetes.io/projected/0e701dd2-07eb-4bad-b5d1-26a1cac38028-kube-api-access-47kbj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.230358 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.230340    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e701dd2-07eb-4bad-b5d1-26a1cac38028-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.230576 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.230442    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.230576 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.230507    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.230576 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.230559    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.331634 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.331593    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.331857 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.331675    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47kbj\" (UniqueName: \"kubernetes.io/projected/0e701dd2-07eb-4bad-b5d1-26a1cac38028-kube-api-access-47kbj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.331857 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.331715    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e701dd2-07eb-4bad-b5d1-26a1cac38028-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.331857 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.331748    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.331857 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.331798    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.331857 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.331840    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.332119 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.332047    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.332119 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.332085    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.332196 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.332168    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.334132 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.334110    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e701dd2-07eb-4bad-b5d1-26a1cac38028-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.334429 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.334410    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e701dd2-07eb-4bad-b5d1-26a1cac38028-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.340203 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.340182    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47kbj\" (UniqueName: \"kubernetes.io/projected/0e701dd2-07eb-4bad-b5d1-26a1cac38028-kube-api-access-47kbj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8k9h\" (UID: \"0e701dd2-07eb-4bad-b5d1-26a1cac38028\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.414003 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.413964    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:39.551315 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.551293    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"]
Jul 09 19:49:39.554792 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:49:39.554762    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e701dd2_07eb_4bad_b5d1_26a1cac38028.slice/crio-a72a51adee080007f45a5fbe961de81c0416648904baf36672a1fb3915945acf WatchSource:0}: Error finding container a72a51adee080007f45a5fbe961de81c0416648904baf36672a1fb3915945acf: Status 404 returned error can't find the container with id a72a51adee080007f45a5fbe961de81c0416648904baf36672a1fb3915945acf
Jul 09 19:49:39.893624 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.893586    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h" event={"ID":"0e701dd2-07eb-4bad-b5d1-26a1cac38028","Type":"ContainerStarted","Data":"7df4ab98251e4dd9d1386dca8be60e44ee7db25b62ac5838fdf02143adf6e7a0"}
Jul 09 19:49:39.893828 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.893703    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h" event={"ID":"0e701dd2-07eb-4bad-b5d1-26a1cac38028","Type":"ContainerStarted","Data":"a72a51adee080007f45a5fbe961de81c0416648904baf36672a1fb3915945acf"}
Jul 09 19:49:39.932230 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:39.932172    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h" podStartSLOduration=0.745318559 podStartE2EDuration="932.154795ms" totalImagesPullingTime="186.836236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:49:39.92816465 +0000 UTC m=+591.329299461" watchObservedRunningTime="2026-07-09 19:49:39.932154795 +0000 UTC m=+591.333289605"
Jul 09 19:49:49.414912 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:49.414877    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:49.415387 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:49.414944    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:49.427635 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:49.427610    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:49:49.950194 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:49:49.950164    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8k9h"
Jul 09 19:57:46.024920 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:46.024835    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"]
Jul 09 19:57:46.027667 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:46.025037    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4" podUID="e47d906f-6517-4b5c-a814-cabce5a146f1" containerName="manager" containerID="cri-o://cb92e46cb050db345596a6a1169214345f35942d3a9b6f9ab3691834fe63ba7d" gracePeriod=10
Jul 09 19:57:48.874176 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:48.874143    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:57:48.966957 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:48.966874    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djnnd\" (UniqueName: \"kubernetes.io/projected/e47d906f-6517-4b5c-a814-cabce5a146f1-kube-api-access-djnnd\") pod \"e47d906f-6517-4b5c-a814-cabce5a146f1\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") "
Jul 09 19:57:48.966957 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:48.966924    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e47d906f-6517-4b5c-a814-cabce5a146f1-extensions-socket-volume\") pod \"e47d906f-6517-4b5c-a814-cabce5a146f1\" (UID: \"e47d906f-6517-4b5c-a814-cabce5a146f1\") "
Jul 09 19:57:48.967334 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:48.967306    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e47d906f-6517-4b5c-a814-cabce5a146f1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e47d906f-6517-4b5c-a814-cabce5a146f1" (UID: "e47d906f-6517-4b5c-a814-cabce5a146f1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:57:48.969020 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:48.968993    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e47d906f-6517-4b5c-a814-cabce5a146f1-kube-api-access-djnnd" (OuterVolumeSpecName: "kube-api-access-djnnd") pod "e47d906f-6517-4b5c-a814-cabce5a146f1" (UID: "e47d906f-6517-4b5c-a814-cabce5a146f1"). InnerVolumeSpecName "kube-api-access-djnnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:57:49.058758 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.058728    2527 scope.go:117] "RemoveContainer" containerID="cb92e46cb050db345596a6a1169214345f35942d3a9b6f9ab3691834fe63ba7d"
Jul 09 19:57:49.067907 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.067877    2527 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e47d906f-6517-4b5c-a814-cabce5a146f1-extensions-socket-volume\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:57:49.068011 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.067911    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djnnd\" (UniqueName: \"kubernetes.io/projected/e47d906f-6517-4b5c-a814-cabce5a146f1-kube-api-access-djnnd\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 19:57:49.265331 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.265248    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"
Jul 09 19:57:49.265475 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.265251    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4" event={"ID":"e47d906f-6517-4b5c-a814-cabce5a146f1","Type":"ContainerDied","Data":"cb92e46cb050db345596a6a1169214345f35942d3a9b6f9ab3691834fe63ba7d"}
Jul 09 19:57:49.265475 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.265385    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4" event={"ID":"e47d906f-6517-4b5c-a814-cabce5a146f1","Type":"ContainerDied","Data":"48fc3aed2c725b244f95fde99f8c64a29ec4c0863d181c896fe21c9e269977d9"}
Jul 09 19:57:49.296097 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.296060    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"]
Jul 09 19:57:49.299274 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:49.299250    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nbrw4"]
Jul 09 19:57:51.077379 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:57:51.077348    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e47d906f-6517-4b5c-a814-cabce5a146f1" path="/var/lib/kubelet/pods/e47d906f-6517-4b5c-a814-cabce5a146f1/volumes"
Jul 09 19:58:47.175432 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.175394    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"]
Jul 09 19:58:47.175926 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.175783    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e47d906f-6517-4b5c-a814-cabce5a146f1" containerName="manager"
Jul 09 19:58:47.175926 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.175794    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="e47d906f-6517-4b5c-a814-cabce5a146f1" containerName="manager"
Jul 09 19:58:47.175926 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.175872    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="e47d906f-6517-4b5c-a814-cabce5a146f1" containerName="manager"
Jul 09 19:58:47.178829 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.178813    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.181828 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.181805    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 19:58:47.181828 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.181816    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 19:58:47.182013 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.181816    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jvqwt\""
Jul 09 19:58:47.192398 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.192366    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"]
Jul 09 19:58:47.243903 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.243870    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56875306-23fc-4064-8c01-8e5c96de29f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d4s7g\" (UID: \"56875306-23fc-4064-8c01-8e5c96de29f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.244099 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.243957    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq4hv\" (UniqueName: \"kubernetes.io/projected/56875306-23fc-4064-8c01-8e5c96de29f2-kube-api-access-vq4hv\") pod \"kuadrant-operator-controller-manager-86878c769f-d4s7g\" (UID: \"56875306-23fc-4064-8c01-8e5c96de29f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.345498 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.345459    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56875306-23fc-4064-8c01-8e5c96de29f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d4s7g\" (UID: \"56875306-23fc-4064-8c01-8e5c96de29f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.345698 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.345537    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq4hv\" (UniqueName: \"kubernetes.io/projected/56875306-23fc-4064-8c01-8e5c96de29f2-kube-api-access-vq4hv\") pod \"kuadrant-operator-controller-manager-86878c769f-d4s7g\" (UID: \"56875306-23fc-4064-8c01-8e5c96de29f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.345895 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.345877    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56875306-23fc-4064-8c01-8e5c96de29f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d4s7g\" (UID: \"56875306-23fc-4064-8c01-8e5c96de29f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.359732 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.359710    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq4hv\" (UniqueName: \"kubernetes.io/projected/56875306-23fc-4064-8c01-8e5c96de29f2-kube-api-access-vq4hv\") pod \"kuadrant-operator-controller-manager-86878c769f-d4s7g\" (UID: \"56875306-23fc-4064-8c01-8e5c96de29f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.490023 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.489928    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:47.634625 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:47.634599    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"]
Jul 09 19:58:47.637416 ip-10-0-136-128 kubenswrapper[2527]: W0709 19:58:47.637385    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56875306_23fc_4064_8c01_8e5c96de29f2.slice/crio-a0d382f4aeba1f5a26ba380004ab5852af9dad55c59bd1db01262e1f40c906c7 WatchSource:0}: Error finding container a0d382f4aeba1f5a26ba380004ab5852af9dad55c59bd1db01262e1f40c906c7: Status 404 returned error can't find the container with id a0d382f4aeba1f5a26ba380004ab5852af9dad55c59bd1db01262e1f40c906c7
Jul 09 19:58:48.552829 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:48.552795    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g" event={"ID":"56875306-23fc-4064-8c01-8e5c96de29f2","Type":"ContainerStarted","Data":"88f009d72ce77f444ba848cffe291d5bd900eb9c06ae222b6e55352048f2e8af"}
Jul 09 19:58:48.553214 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:48.552846    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g" event={"ID":"56875306-23fc-4064-8c01-8e5c96de29f2","Type":"ContainerStarted","Data":"a0d382f4aeba1f5a26ba380004ab5852af9dad55c59bd1db01262e1f40c906c7"}
Jul 09 19:58:48.553214 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:48.552892    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 19:58:48.578019 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:48.577959    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g" podStartSLOduration=1.577937626 podStartE2EDuration="1.577937626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:58:48.576474646 +0000 UTC m=+1139.977609481" watchObservedRunningTime="2026-07-09 19:58:48.577937626 +0000 UTC m=+1139.979072437"
Jul 09 19:58:59.560554 ip-10-0-136-128 kubenswrapper[2527]: I0709 19:58:59.560516    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d4s7g"
Jul 09 20:05:13.465891 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.465855    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"]
Jul 09 20:05:13.469780 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.469758    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.472561 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.472533    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-581685c1-serving-cert\""
Jul 09 20:05:13.472716 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.472578    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:05:13.472716 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.472536    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:05:13.473719 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.473692    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:05:13.480442 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.480413    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"]
Jul 09 20:05:13.633530 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.633490    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knmjv\" (UniqueName: \"kubernetes.io/projected/aeb89660-4817-4fe4-8644-268d2dbbf189-kube-api-access-knmjv\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.633530 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.633535    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb89660-4817-4fe4-8644-268d2dbbf189-openshift-service-ca\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.633791 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.633628    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb89660-4817-4fe4-8644-268d2dbbf189-maas-api-tls\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.734696 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.734581    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knmjv\" (UniqueName: \"kubernetes.io/projected/aeb89660-4817-4fe4-8644-268d2dbbf189-kube-api-access-knmjv\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.734696 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.734623    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb89660-4817-4fe4-8644-268d2dbbf189-openshift-service-ca\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.734884 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.734751    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb89660-4817-4fe4-8644-268d2dbbf189-maas-api-tls\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.735293 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.735273    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb89660-4817-4fe4-8644-268d2dbbf189-openshift-service-ca\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.737227 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.737206    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb89660-4817-4fe4-8644-268d2dbbf189-maas-api-tls\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.745753 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.745695    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knmjv\" (UniqueName: \"kubernetes.io/projected/aeb89660-4817-4fe4-8644-268d2dbbf189-kube-api-access-knmjv\") pod \"maas-api-e2e-ait-581685c1-f6bf8df96-rqml7\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.783534 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.783497    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:13.922354 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.922328    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"]
Jul 09 20:05:13.924632 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:05:13.924599    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb89660_4817_4fe4_8644_268d2dbbf189.slice/crio-9134062a9b5ef39eb719eca8fcc0d3b90f0ea1051c2e7f168c649f3d2ae3774e WatchSource:0}: Error finding container 9134062a9b5ef39eb719eca8fcc0d3b90f0ea1051c2e7f168c649f3d2ae3774e: Status 404 returned error can't find the container with id 9134062a9b5ef39eb719eca8fcc0d3b90f0ea1051c2e7f168c649f3d2ae3774e
Jul 09 20:05:13.925924 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:13.925908    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 20:05:14.450002 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:14.449964    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7" event={"ID":"aeb89660-4817-4fe4-8644-268d2dbbf189","Type":"ContainerStarted","Data":"9134062a9b5ef39eb719eca8fcc0d3b90f0ea1051c2e7f168c649f3d2ae3774e"}
Jul 09 20:05:16.462343 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:16.462298    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7" event={"ID":"aeb89660-4817-4fe4-8644-268d2dbbf189","Type":"ContainerStarted","Data":"cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7"}
Jul 09 20:05:16.462746 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:16.462363    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:05:16.480173 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:16.480108    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7" podStartSLOduration=1.155385696 podStartE2EDuration="3.480086694s" totalImagesPullingTime="2.324700998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:05:16.478724255 +0000 UTC m=+1527.879859067" watchObservedRunningTime="2026-07-09 20:05:16.480086694 +0000 UTC m=+1527.881221505"
Jul 09 20:05:22.472091 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:05:22.472058    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:06:24.460492 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.460394    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"]
Jul 09 20:06:24.460967 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.460739    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7" podUID="aeb89660-4817-4fe4-8644-268d2dbbf189" containerName="maas-api" containerID="cri-o://cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7" gracePeriod=30
Jul 09 20:06:24.709884 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.709858    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:06:24.807074 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.807037    2527 generic.go:356] "Generic (PLEG): container finished" podID="aeb89660-4817-4fe4-8644-268d2dbbf189" containerID="cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7" exitCode=0
Jul 09 20:06:24.807267 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.807089    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"
Jul 09 20:06:24.807267 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.807122    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7" event={"ID":"aeb89660-4817-4fe4-8644-268d2dbbf189","Type":"ContainerDied","Data":"cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7"}
Jul 09 20:06:24.807267 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.807179    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7" event={"ID":"aeb89660-4817-4fe4-8644-268d2dbbf189","Type":"ContainerDied","Data":"9134062a9b5ef39eb719eca8fcc0d3b90f0ea1051c2e7f168c649f3d2ae3774e"}
Jul 09 20:06:24.807267 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.807203    2527 scope.go:117] "RemoveContainer" containerID="cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7"
Jul 09 20:06:24.816080 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.816059    2527 scope.go:117] "RemoveContainer" containerID="cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7"
Jul 09 20:06:24.816343 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:06:24.816324    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7\": container with ID starting with cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7 not found: ID does not exist" containerID="cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7"
Jul 09 20:06:24.816396 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.816352    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7"} err="failed to get container status \"cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7\": rpc error: code = NotFound desc = could not find container \"cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7\": container with ID starting with cdb629a104c2673d7c4b9b42b3dcc3839ba2d7986da049628f1b02f4103efec7 not found: ID does not exist"
Jul 09 20:06:24.835792 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.835767    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knmjv\" (UniqueName: \"kubernetes.io/projected/aeb89660-4817-4fe4-8644-268d2dbbf189-kube-api-access-knmjv\") pod \"aeb89660-4817-4fe4-8644-268d2dbbf189\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") "
Jul 09 20:06:24.835909 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.835823    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb89660-4817-4fe4-8644-268d2dbbf189-openshift-service-ca\") pod \"aeb89660-4817-4fe4-8644-268d2dbbf189\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") "
Jul 09 20:06:24.835909 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.835848    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb89660-4817-4fe4-8644-268d2dbbf189-maas-api-tls\") pod \"aeb89660-4817-4fe4-8644-268d2dbbf189\" (UID: \"aeb89660-4817-4fe4-8644-268d2dbbf189\") "
Jul 09 20:06:24.836136 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.836114    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeb89660-4817-4fe4-8644-268d2dbbf189-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aeb89660-4817-4fe4-8644-268d2dbbf189" (UID: "aeb89660-4817-4fe4-8644-268d2dbbf189"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:06:24.838217 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.838196    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb89660-4817-4fe4-8644-268d2dbbf189-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aeb89660-4817-4fe4-8644-268d2dbbf189" (UID: "aeb89660-4817-4fe4-8644-268d2dbbf189"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:06:24.838217 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.838199    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb89660-4817-4fe4-8644-268d2dbbf189-kube-api-access-knmjv" (OuterVolumeSpecName: "kube-api-access-knmjv") pod "aeb89660-4817-4fe4-8644-268d2dbbf189" (UID: "aeb89660-4817-4fe4-8644-268d2dbbf189"). InnerVolumeSpecName "kube-api-access-knmjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:06:24.936624 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.936594    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knmjv\" (UniqueName: \"kubernetes.io/projected/aeb89660-4817-4fe4-8644-268d2dbbf189-kube-api-access-knmjv\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:06:24.936624 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.936620    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb89660-4817-4fe4-8644-268d2dbbf189-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:06:24.936624 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:24.936630    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb89660-4817-4fe4-8644-268d2dbbf189-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:06:25.125489 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:25.125459    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"]
Jul 09 20:06:25.131814 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:25.131784    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-581685c1-f6bf8df96-rqml7"]
Jul 09 20:06:27.077231 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:27.077195    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb89660-4817-4fe4-8644-268d2dbbf189" path="/var/lib/kubelet/pods/aeb89660-4817-4fe4-8644-268d2dbbf189/volumes"
Jul 09 20:06:31.289757 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.289726    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"]
Jul 09 20:06:31.290309 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.290289    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb89660-4817-4fe4-8644-268d2dbbf189" containerName="maas-api"
Jul 09 20:06:31.290393 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.290312    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb89660-4817-4fe4-8644-268d2dbbf189" containerName="maas-api"
Jul 09 20:06:31.290462 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.290449    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb89660-4817-4fe4-8644-268d2dbbf189" containerName="maas-api"
Jul 09 20:06:31.294983 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.294962    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.297747 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.297721    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8c04d3b2-serving-cert\""
Jul 09 20:06:31.297866 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.297767    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:06:31.299073 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.299039    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:06:31.299466 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.299449    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:06:31.302186 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.302164    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"]
Jul 09 20:06:31.403196 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.403163    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0021b23-e966-44a3-ad0e-f7b434baebdd-openshift-service-ca\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.403380 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.403211    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.403380 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.403323    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfhts\" (UniqueName: \"kubernetes.io/projected/c0021b23-e966-44a3-ad0e-f7b434baebdd-kube-api-access-bfhts\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.504162 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.504124    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfhts\" (UniqueName: \"kubernetes.io/projected/c0021b23-e966-44a3-ad0e-f7b434baebdd-kube-api-access-bfhts\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.504356 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.504197    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0021b23-e966-44a3-ad0e-f7b434baebdd-openshift-service-ca\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.504356 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.504229    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.504356 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:06:31.504345    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-serving-cert: secret "maas-api-e2e-ait-8c04d3b2-serving-cert" not found
Jul 09 20:06:31.504518 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:06:31.504408    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls podName:c0021b23-e966-44a3-ad0e-f7b434baebdd nodeName:}" failed. No retries permitted until 2026-07-09 20:06:32.00438674 +0000 UTC m=+1603.405521527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls") pod "maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg" (UID: "c0021b23-e966-44a3-ad0e-f7b434baebdd") : secret "maas-api-e2e-ait-8c04d3b2-serving-cert" not found
Jul 09 20:06:31.504941 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.504921    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0021b23-e966-44a3-ad0e-f7b434baebdd-openshift-service-ca\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:31.513203 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:31.513174    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfhts\" (UniqueName: \"kubernetes.io/projected/c0021b23-e966-44a3-ad0e-f7b434baebdd-kube-api-access-bfhts\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:32.009525 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:32.009487    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:32.012202 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:32.012176    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls\") pod \"maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:32.208576 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:32.208543    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:32.364614 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:32.364501    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"]
Jul 09 20:06:32.367486 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:06:32.367461    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0021b23_e966_44a3_ad0e_f7b434baebdd.slice/crio-1529730b2e27e7ce52b206791271e98d186937ce45c7f02df1782509e6fbe77c WatchSource:0}: Error finding container 1529730b2e27e7ce52b206791271e98d186937ce45c7f02df1782509e6fbe77c: Status 404 returned error can't find the container with id 1529730b2e27e7ce52b206791271e98d186937ce45c7f02df1782509e6fbe77c
Jul 09 20:06:32.847504 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:32.847471    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg" event={"ID":"c0021b23-e966-44a3-ad0e-f7b434baebdd","Type":"ContainerStarted","Data":"1529730b2e27e7ce52b206791271e98d186937ce45c7f02df1782509e6fbe77c"}
Jul 09 20:06:33.854023 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:33.853934    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg" event={"ID":"c0021b23-e966-44a3-ad0e-f7b434baebdd","Type":"ContainerStarted","Data":"17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220"}
Jul 09 20:06:33.854411 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:33.854070    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:33.872093 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:33.872040    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg" podStartSLOduration=2.302713417 podStartE2EDuration="2.872024563s" totalImagesPullingTime="569.311146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:06:33.869995916 +0000 UTC m=+1605.271130726" watchObservedRunningTime="2026-07-09 20:06:33.872024563 +0000 UTC m=+1605.273159408"
Jul 09 20:06:36.379712 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.379672    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"]
Jul 09 20:06:36.383524 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.383501    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"
Jul 09 20:06:36.388304 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.387077    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:06:36.389956 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.389926    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"]
Jul 09 20:06:36.451015 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.450982    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db29c68b-385a-4b42-80dd-fe1886bf38e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5\" (UID: \"db29c68b-385a-4b42-80dd-fe1886bf38e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"
Jul 09 20:06:36.551930 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.551895    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db29c68b-385a-4b42-80dd-fe1886bf38e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5\" (UID: \"db29c68b-385a-4b42-80dd-fe1886bf38e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"
Jul 09 20:06:36.552519 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.552500    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db29c68b-385a-4b42-80dd-fe1886bf38e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5\" (UID: \"db29c68b-385a-4b42-80dd-fe1886bf38e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"
Jul 09 20:06:36.696418 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.696323    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"
Jul 09 20:06:36.832218 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.832191    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"]
Jul 09 20:06:36.833511 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:06:36.833479    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb29c68b_385a_4b42_80dd_fe1886bf38e6.slice/crio-ff10a4de9cf24c7fd65bd9a22e6b0bf43b13ee92828b184ea4e3f899f86fcc68 WatchSource:0}: Error finding container ff10a4de9cf24c7fd65bd9a22e6b0bf43b13ee92828b184ea4e3f899f86fcc68: Status 404 returned error can't find the container with id ff10a4de9cf24c7fd65bd9a22e6b0bf43b13ee92828b184ea4e3f899f86fcc68
Jul 09 20:06:36.871829 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:36.871797    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5" event={"ID":"db29c68b-385a-4b42-80dd-fe1886bf38e6","Type":"ContainerStarted","Data":"ff10a4de9cf24c7fd65bd9a22e6b0bf43b13ee92828b184ea4e3f899f86fcc68"}
Jul 09 20:06:39.865344 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:39.865262    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:06:58.982532 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:58.982488    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5" event={"ID":"db29c68b-385a-4b42-80dd-fe1886bf38e6","Type":"ContainerStarted","Data":"fcc84c99e9a43e7d796cb0a41a38ce5e47a5927cdc2e6c8e0d8801ddf605bad1"}
Jul 09 20:06:58.997412 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:58.997358    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5" podStartSLOduration=1.504344554 podStartE2EDuration="22.99733623s" totalImagesPullingTime="21.492991676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:06:58.996990705 +0000 UTC m=+1630.398125515" watchObservedRunningTime="2026-07-09 20:06:58.99733623 +0000 UTC m=+1630.398471041"
Jul 09 20:06:59.988787 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:59.988753    2527 generic.go:356] "Generic (PLEG): container finished" podID="db29c68b-385a-4b42-80dd-fe1886bf38e6" containerID="fcc84c99e9a43e7d796cb0a41a38ce5e47a5927cdc2e6c8e0d8801ddf605bad1" exitCode=0
Jul 09 20:06:59.989192 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:06:59.988793    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5" event={"ID":"db29c68b-385a-4b42-80dd-fe1886bf38e6","Type":"ContainerDied","Data":"fcc84c99e9a43e7d796cb0a41a38ce5e47a5927cdc2e6c8e0d8801ddf605bad1"}
Jul 09 20:07:01.129604 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:01.129580    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"
Jul 09 20:07:01.302837 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:01.302803    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db29c68b-385a-4b42-80dd-fe1886bf38e6-maas-api-service-ca\") pod \"db29c68b-385a-4b42-80dd-fe1886bf38e6\" (UID: \"db29c68b-385a-4b42-80dd-fe1886bf38e6\") "
Jul 09 20:07:01.303170 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:01.303145    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db29c68b-385a-4b42-80dd-fe1886bf38e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "db29c68b-385a-4b42-80dd-fe1886bf38e6" (UID: "db29c68b-385a-4b42-80dd-fe1886bf38e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:07:01.403706 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:01.403673    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db29c68b-385a-4b42-80dd-fe1886bf38e6-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:07:02.001234 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:02.001199    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5" event={"ID":"db29c68b-385a-4b42-80dd-fe1886bf38e6","Type":"ContainerDied","Data":"ff10a4de9cf24c7fd65bd9a22e6b0bf43b13ee92828b184ea4e3f899f86fcc68"}
Jul 09 20:07:02.001234 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:02.001236    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5"
Jul 09 20:07:02.001441 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:02.001255    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff10a4de9cf24c7fd65bd9a22e6b0bf43b13ee92828b184ea4e3f899f86fcc68"
Jul 09 20:07:06.751593 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:06.751553    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"]
Jul 09 20:07:06.752100 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:06.751838    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg" podUID="c0021b23-e966-44a3-ad0e-f7b434baebdd" containerName="maas-api" containerID="cri-o://17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220" gracePeriod=30
Jul 09 20:07:07.012693 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.012604    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:07:07.026088 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.026053    2527 generic.go:356] "Generic (PLEG): container finished" podID="c0021b23-e966-44a3-ad0e-f7b434baebdd" containerID="17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220" exitCode=0
Jul 09 20:07:07.026225 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.026095    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"
Jul 09 20:07:07.026225 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.026134    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg" event={"ID":"c0021b23-e966-44a3-ad0e-f7b434baebdd","Type":"ContainerDied","Data":"17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220"}
Jul 09 20:07:07.026225 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.026180    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg" event={"ID":"c0021b23-e966-44a3-ad0e-f7b434baebdd","Type":"ContainerDied","Data":"1529730b2e27e7ce52b206791271e98d186937ce45c7f02df1782509e6fbe77c"}
Jul 09 20:07:07.026225 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.026208    2527 scope.go:117] "RemoveContainer" containerID="17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220"
Jul 09 20:07:07.036715 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.036696    2527 scope.go:117] "RemoveContainer" containerID="17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220"
Jul 09 20:07:07.037004 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:07:07.036975    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220\": container with ID starting with 17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220 not found: ID does not exist" containerID="17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220"
Jul 09 20:07:07.037054 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.037019    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220"} err="failed to get container status \"17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220\": rpc error: code = NotFound desc = could not find container \"17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220\": container with ID starting with 17068b8b496e44d04f7c16a8cce56aa555b5daed2cbc16f0f0bbf31baf58d220 not found: ID does not exist"
Jul 09 20:07:07.055531 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.055497    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0021b23-e966-44a3-ad0e-f7b434baebdd-openshift-service-ca\") pod \"c0021b23-e966-44a3-ad0e-f7b434baebdd\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") "
Jul 09 20:07:07.055674 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.055624    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls\") pod \"c0021b23-e966-44a3-ad0e-f7b434baebdd\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") "
Jul 09 20:07:07.055737 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.055721    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfhts\" (UniqueName: \"kubernetes.io/projected/c0021b23-e966-44a3-ad0e-f7b434baebdd-kube-api-access-bfhts\") pod \"c0021b23-e966-44a3-ad0e-f7b434baebdd\" (UID: \"c0021b23-e966-44a3-ad0e-f7b434baebdd\") "
Jul 09 20:07:07.055872 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.055851    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0021b23-e966-44a3-ad0e-f7b434baebdd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0021b23-e966-44a3-ad0e-f7b434baebdd" (UID: "c0021b23-e966-44a3-ad0e-f7b434baebdd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:07:07.056028 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.056011    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0021b23-e966-44a3-ad0e-f7b434baebdd-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:07:07.057818 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.057798    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0021b23-e966-44a3-ad0e-f7b434baebdd" (UID: "c0021b23-e966-44a3-ad0e-f7b434baebdd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:07:07.057994 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.057974    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0021b23-e966-44a3-ad0e-f7b434baebdd-kube-api-access-bfhts" (OuterVolumeSpecName: "kube-api-access-bfhts") pod "c0021b23-e966-44a3-ad0e-f7b434baebdd" (UID: "c0021b23-e966-44a3-ad0e-f7b434baebdd"). InnerVolumeSpecName "kube-api-access-bfhts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:07:07.156914 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.156877    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0021b23-e966-44a3-ad0e-f7b434baebdd-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:07:07.156914 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.156910    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfhts\" (UniqueName: \"kubernetes.io/projected/c0021b23-e966-44a3-ad0e-f7b434baebdd-kube-api-access-bfhts\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:07:07.342493 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.342460    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"]
Jul 09 20:07:07.346100 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:07.346072    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c04d3b2-5758fc7d7f-lqkwg"]
Jul 09 20:07:09.077168 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:09.077130    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0021b23-e966-44a3-ad0e-f7b434baebdd" path="/var/lib/kubelet/pods/c0021b23-e966-44a3-ad0e-f7b434baebdd/volumes"
Jul 09 20:07:24.200045 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.200015    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"]
Jul 09 20:07:24.200497 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.200480    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db29c68b-385a-4b42-80dd-fe1886bf38e6" containerName="revoke-keys"
Jul 09 20:07:24.200538 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.200499    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="db29c68b-385a-4b42-80dd-fe1886bf38e6" containerName="revoke-keys"
Jul 09 20:07:24.200538 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.200518    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0021b23-e966-44a3-ad0e-f7b434baebdd" containerName="maas-api"
Jul 09 20:07:24.200538 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.200524    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0021b23-e966-44a3-ad0e-f7b434baebdd" containerName="maas-api"
Jul 09 20:07:24.200632 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.200603    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0021b23-e966-44a3-ad0e-f7b434baebdd" containerName="maas-api"
Jul 09 20:07:24.200632 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.200614    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="db29c68b-385a-4b42-80dd-fe1886bf38e6" containerName="revoke-keys"
Jul 09 20:07:24.203808 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.203788    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.206713 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.206687    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-e2f7b0b4-serving-cert\""
Jul 09 20:07:24.206713 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.206704    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:07:24.206890 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.206753    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:07:24.207942 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.207921    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:07:24.214457 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.214438    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"]
Jul 09 20:07:24.328557 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.328525    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd15740-ee08-4a60-9aed-197c11abcb20-openshift-service-ca\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.328732 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.328575    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd15740-ee08-4a60-9aed-197c11abcb20-maas-api-tls\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.328732 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.328632    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltvg2\" (UniqueName: \"kubernetes.io/projected/7bd15740-ee08-4a60-9aed-197c11abcb20-kube-api-access-ltvg2\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.429325 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.429290    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltvg2\" (UniqueName: \"kubernetes.io/projected/7bd15740-ee08-4a60-9aed-197c11abcb20-kube-api-access-ltvg2\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.429497 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.429373    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd15740-ee08-4a60-9aed-197c11abcb20-openshift-service-ca\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.429497 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.429410    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd15740-ee08-4a60-9aed-197c11abcb20-maas-api-tls\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.430111 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.430082    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd15740-ee08-4a60-9aed-197c11abcb20-openshift-service-ca\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.431993 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.431975    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd15740-ee08-4a60-9aed-197c11abcb20-maas-api-tls\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.437035 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.437018    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltvg2\" (UniqueName: \"kubernetes.io/projected/7bd15740-ee08-4a60-9aed-197c11abcb20-kube-api-access-ltvg2\") pod \"maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.516279 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.516247    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:24.652150 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:24.652124    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"]
Jul 09 20:07:24.654399 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:07:24.654369    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd15740_ee08_4a60_9aed_197c11abcb20.slice/crio-58aca24675b28c6bff30c72bf2391bdbd1acc0382c6680573285962f298f4d30 WatchSource:0}: Error finding container 58aca24675b28c6bff30c72bf2391bdbd1acc0382c6680573285962f298f4d30: Status 404 returned error can't find the container with id 58aca24675b28c6bff30c72bf2391bdbd1acc0382c6680573285962f298f4d30
Jul 09 20:07:25.115977 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:25.115941    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4" event={"ID":"7bd15740-ee08-4a60-9aed-197c11abcb20","Type":"ContainerStarted","Data":"58aca24675b28c6bff30c72bf2391bdbd1acc0382c6680573285962f298f4d30"}
Jul 09 20:07:26.122511 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:26.122479    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4" event={"ID":"7bd15740-ee08-4a60-9aed-197c11abcb20","Type":"ContainerStarted","Data":"afd940c23ff23ad3fd3b17d25cd200d024b0d13dcec8b65523338258fafacaf7"}
Jul 09 20:07:26.122920 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:26.122598    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:26.139606 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:26.139560    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4" podStartSLOduration=1.738617546 podStartE2EDuration="2.139544135s" totalImagesPullingTime="400.926589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:07:26.137460605 +0000 UTC m=+1657.538595412" watchObservedRunningTime="2026-07-09 20:07:26.139544135 +0000 UTC m=+1657.540678961"
Jul 09 20:07:28.378681 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.378630    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"]
Jul 09 20:07:28.384220 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.384203    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"
Jul 09 20:07:28.387205 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.387177    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:07:28.389546 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.389512    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"]
Jul 09 20:07:28.473682 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.473606    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a461a6-59c3-4909-82e2-7157d7b73484-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l\" (UID: \"d4a461a6-59c3-4909-82e2-7157d7b73484\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"
Jul 09 20:07:28.575135 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.575078    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a461a6-59c3-4909-82e2-7157d7b73484-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l\" (UID: \"d4a461a6-59c3-4909-82e2-7157d7b73484\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"
Jul 09 20:07:28.575810 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.575791    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a461a6-59c3-4909-82e2-7157d7b73484-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l\" (UID: \"d4a461a6-59c3-4909-82e2-7157d7b73484\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"
Jul 09 20:07:28.697252 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.697173    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"
Jul 09 20:07:28.833153 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:28.833089    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"]
Jul 09 20:07:28.835503 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:07:28.835479    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a461a6_59c3_4909_82e2_7157d7b73484.slice/crio-da0e643691470805601d6f60bb639518b22fdcaa5f9d3949a92fb14541fb37bb WatchSource:0}: Error finding container da0e643691470805601d6f60bb639518b22fdcaa5f9d3949a92fb14541fb37bb: Status 404 returned error can't find the container with id da0e643691470805601d6f60bb639518b22fdcaa5f9d3949a92fb14541fb37bb
Jul 09 20:07:29.136970 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:29.136933    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l" event={"ID":"d4a461a6-59c3-4909-82e2-7157d7b73484","Type":"ContainerStarted","Data":"da0e643691470805601d6f60bb639518b22fdcaa5f9d3949a92fb14541fb37bb"}
Jul 09 20:07:30.143046 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:30.143007    2527 generic.go:356] "Generic (PLEG): container finished" podID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerID="c8465dc2abd71456adb606dbae8c64068b121920c9aadf5ff2849255857ff482" exitCode=7
Jul 09 20:07:30.143484 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:30.143098    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l" event={"ID":"d4a461a6-59c3-4909-82e2-7157d7b73484","Type":"ContainerDied","Data":"c8465dc2abd71456adb606dbae8c64068b121920c9aadf5ff2849255857ff482"}
Jul 09 20:07:30.143484 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:30.143285    2527 scope.go:117] "RemoveContainer" containerID="c8465dc2abd71456adb606dbae8c64068b121920c9aadf5ff2849255857ff482"
Jul 09 20:07:31.150300 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:31.150210    2527 generic.go:356] "Generic (PLEG): container finished" podID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerID="5b51ef0a676fab50204a70c3fbd2db4bbc1a7fe12c432bc6c763b3d995157865" exitCode=7
Jul 09 20:07:31.150757 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:31.150307    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l" event={"ID":"d4a461a6-59c3-4909-82e2-7157d7b73484","Type":"ContainerDied","Data":"5b51ef0a676fab50204a70c3fbd2db4bbc1a7fe12c432bc6c763b3d995157865"}
Jul 09 20:07:31.150757 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:31.150373    2527 scope.go:117] "RemoveContainer" containerID="c8465dc2abd71456adb606dbae8c64068b121920c9aadf5ff2849255857ff482"
Jul 09 20:07:31.150757 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:31.150476    2527 scope.go:117] "RemoveContainer" containerID="5b51ef0a676fab50204a70c3fbd2db4bbc1a7fe12c432bc6c763b3d995157865"
Jul 09 20:07:31.150869 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:07:31.150754    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l_odh-ai-gateway-infra(d4a461a6-59c3-4909-82e2-7157d7b73484)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484"
Jul 09 20:07:32.132493 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:32.132462    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:07:32.157625 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:32.157577    2527 scope.go:117] "RemoveContainer" containerID="5b51ef0a676fab50204a70c3fbd2db4bbc1a7fe12c432bc6c763b3d995157865"
Jul 09 20:07:32.158076 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:07:32.157874    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l_odh-ai-gateway-infra(d4a461a6-59c3-4909-82e2-7157d7b73484)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484"
Jul 09 20:07:47.072046 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:47.072016    2527 scope.go:117] "RemoveContainer" containerID="5b51ef0a676fab50204a70c3fbd2db4bbc1a7fe12c432bc6c763b3d995157865"
Jul 09 20:07:48.245787 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:48.245696    2527 generic.go:356] "Generic (PLEG): container finished" podID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerID="c6d6454e07def1eee09538b7ce93e1a40255c8ba68e538dd4fc5305eea57d423" exitCode=0
Jul 09 20:07:48.245787 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:48.245760    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l" event={"ID":"d4a461a6-59c3-4909-82e2-7157d7b73484","Type":"ContainerDied","Data":"c6d6454e07def1eee09538b7ce93e1a40255c8ba68e538dd4fc5305eea57d423"}
Jul 09 20:07:48.246216 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:48.245826    2527 scope.go:117] "RemoveContainer" containerID="5b51ef0a676fab50204a70c3fbd2db4bbc1a7fe12c432bc6c763b3d995157865"
Jul 09 20:07:49.277690 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:49.277632    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"]
Jul 09 20:07:49.390443 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:49.390414    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"
Jul 09 20:07:49.475193 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:49.475108    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a461a6-59c3-4909-82e2-7157d7b73484-maas-api-service-ca\") pod \"d4a461a6-59c3-4909-82e2-7157d7b73484\" (UID: \"d4a461a6-59c3-4909-82e2-7157d7b73484\") "
Jul 09 20:07:49.475470 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:49.475446    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a461a6-59c3-4909-82e2-7157d7b73484-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4a461a6-59c3-4909-82e2-7157d7b73484" (UID: "d4a461a6-59c3-4909-82e2-7157d7b73484"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:07:49.575842 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:49.575804    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a461a6-59c3-4909-82e2-7157d7b73484-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:07:50.257564 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:50.257537    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"
Jul 09 20:07:50.257763 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:50.257572    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l" event={"ID":"d4a461a6-59c3-4909-82e2-7157d7b73484","Type":"ContainerDied","Data":"da0e643691470805601d6f60bb639518b22fdcaa5f9d3949a92fb14541fb37bb"}
Jul 09 20:07:50.257763 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:50.257620    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da0e643691470805601d6f60bb639518b22fdcaa5f9d3949a92fb14541fb37bb"
Jul 09 20:07:50.287517 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:50.287484    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"]
Jul 09 20:07:50.289427 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:50.289402    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-bk78l"]
Jul 09 20:07:51.077035 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:07:51.076992    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" path="/var/lib/kubelet/pods/d4a461a6-59c3-4909-82e2-7157d7b73484/volumes"
Jul 09 20:08:28.426207 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426172    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"]
Jul 09 20:08:28.426695 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426598    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:08:28.426695 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426610    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:08:28.426695 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426625    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:08:28.426695 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426631    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:08:28.426845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426738    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:08:28.426845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426747    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:08:28.426845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.426755    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:08:28.429935 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.429918    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"
Jul 09 20:08:28.432609 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.432581    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:08:28.437004 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.436978    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"]
Jul 09 20:08:28.460043 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.460008    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a5bf184-da9a-411f-acd1-9f5668e88995-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t\" (UID: \"0a5bf184-da9a-411f-acd1-9f5668e88995\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"
Jul 09 20:08:28.561176 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.561143    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a5bf184-da9a-411f-acd1-9f5668e88995-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t\" (UID: \"0a5bf184-da9a-411f-acd1-9f5668e88995\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"
Jul 09 20:08:28.561840 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.561819    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a5bf184-da9a-411f-acd1-9f5668e88995-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t\" (UID: \"0a5bf184-da9a-411f-acd1-9f5668e88995\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"
Jul 09 20:08:28.741962 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.741874    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"
Jul 09 20:08:28.879452 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:28.879415    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"]
Jul 09 20:08:28.881725 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:08:28.881686    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5bf184_da9a_411f_acd1_9f5668e88995.slice/crio-b7b335894121dc4bae57433cf0ce405624b1661b0ed3fd2526bf15e43781e369 WatchSource:0}: Error finding container b7b335894121dc4bae57433cf0ce405624b1661b0ed3fd2526bf15e43781e369: Status 404 returned error can't find the container with id b7b335894121dc4bae57433cf0ce405624b1661b0ed3fd2526bf15e43781e369
Jul 09 20:08:29.448781 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:29.448681    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t" event={"ID":"0a5bf184-da9a-411f-acd1-9f5668e88995","Type":"ContainerStarted","Data":"fa5855a7e20ed5ab0976465ea5b322d77cbf6e0d8448e5dac065100c8d874f9b"}
Jul 09 20:08:29.448781 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:29.448747    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t" event={"ID":"0a5bf184-da9a-411f-acd1-9f5668e88995","Type":"ContainerStarted","Data":"b7b335894121dc4bae57433cf0ce405624b1661b0ed3fd2526bf15e43781e369"}
Jul 09 20:08:29.464224 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:29.464178    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t" podStartSLOduration=1.4641610809999999 podStartE2EDuration="1.464161081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:08:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:08:29.462632199 +0000 UTC m=+1720.863767010" watchObservedRunningTime="2026-07-09 20:08:29.464161081 +0000 UTC m=+1720.865295891"
Jul 09 20:08:30.455362 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:30.455320    2527 generic.go:356] "Generic (PLEG): container finished" podID="0a5bf184-da9a-411f-acd1-9f5668e88995" containerID="fa5855a7e20ed5ab0976465ea5b322d77cbf6e0d8448e5dac065100c8d874f9b" exitCode=0
Jul 09 20:08:30.455852 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:30.455365    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t" event={"ID":"0a5bf184-da9a-411f-acd1-9f5668e88995","Type":"ContainerDied","Data":"fa5855a7e20ed5ab0976465ea5b322d77cbf6e0d8448e5dac065100c8d874f9b"}
Jul 09 20:08:31.591966 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:31.591941    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"
Jul 09 20:08:31.691864 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:31.691827    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a5bf184-da9a-411f-acd1-9f5668e88995-maas-api-service-ca\") pod \"0a5bf184-da9a-411f-acd1-9f5668e88995\" (UID: \"0a5bf184-da9a-411f-acd1-9f5668e88995\") "
Jul 09 20:08:31.692194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:31.692171    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a5bf184-da9a-411f-acd1-9f5668e88995-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a5bf184-da9a-411f-acd1-9f5668e88995" (UID: "0a5bf184-da9a-411f-acd1-9f5668e88995"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:08:31.793394 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:31.793360    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a5bf184-da9a-411f-acd1-9f5668e88995-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:08:32.465909 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:32.465874    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t"
Jul 09 20:08:32.466099 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:32.465913    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t" event={"ID":"0a5bf184-da9a-411f-acd1-9f5668e88995","Type":"ContainerDied","Data":"b7b335894121dc4bae57433cf0ce405624b1661b0ed3fd2526bf15e43781e369"}
Jul 09 20:08:32.466099 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:32.465958    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7b335894121dc4bae57433cf0ce405624b1661b0ed3fd2526bf15e43781e369"
Jul 09 20:08:34.359446 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.359414    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"]
Jul 09 20:08:34.359871 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.359674    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4" podUID="7bd15740-ee08-4a60-9aed-197c11abcb20" containerName="maas-api" containerID="cri-o://afd940c23ff23ad3fd3b17d25cd200d024b0d13dcec8b65523338258fafacaf7" gracePeriod=30
Jul 09 20:08:34.477490 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.477457    2527 generic.go:356] "Generic (PLEG): container finished" podID="7bd15740-ee08-4a60-9aed-197c11abcb20" containerID="afd940c23ff23ad3fd3b17d25cd200d024b0d13dcec8b65523338258fafacaf7" exitCode=0
Jul 09 20:08:34.477628 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.477538    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4" event={"ID":"7bd15740-ee08-4a60-9aed-197c11abcb20","Type":"ContainerDied","Data":"afd940c23ff23ad3fd3b17d25cd200d024b0d13dcec8b65523338258fafacaf7"}
Jul 09 20:08:34.613583 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.613519    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:08:34.720685 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.720613    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd15740-ee08-4a60-9aed-197c11abcb20-openshift-service-ca\") pod \"7bd15740-ee08-4a60-9aed-197c11abcb20\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") "
Jul 09 20:08:34.720685 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.720689    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltvg2\" (UniqueName: \"kubernetes.io/projected/7bd15740-ee08-4a60-9aed-197c11abcb20-kube-api-access-ltvg2\") pod \"7bd15740-ee08-4a60-9aed-197c11abcb20\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") "
Jul 09 20:08:34.720941 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.720761    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd15740-ee08-4a60-9aed-197c11abcb20-maas-api-tls\") pod \"7bd15740-ee08-4a60-9aed-197c11abcb20\" (UID: \"7bd15740-ee08-4a60-9aed-197c11abcb20\") "
Jul 09 20:08:34.721043 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.721003    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bd15740-ee08-4a60-9aed-197c11abcb20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7bd15740-ee08-4a60-9aed-197c11abcb20" (UID: "7bd15740-ee08-4a60-9aed-197c11abcb20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:08:34.721170 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.721124    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bd15740-ee08-4a60-9aed-197c11abcb20-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:08:34.723012 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.722986    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bd15740-ee08-4a60-9aed-197c11abcb20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7bd15740-ee08-4a60-9aed-197c11abcb20" (UID: "7bd15740-ee08-4a60-9aed-197c11abcb20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:08:34.723095 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.723043    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd15740-ee08-4a60-9aed-197c11abcb20-kube-api-access-ltvg2" (OuterVolumeSpecName: "kube-api-access-ltvg2") pod "7bd15740-ee08-4a60-9aed-197c11abcb20" (UID: "7bd15740-ee08-4a60-9aed-197c11abcb20"). InnerVolumeSpecName "kube-api-access-ltvg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:08:34.821749 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.821711    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltvg2\" (UniqueName: \"kubernetes.io/projected/7bd15740-ee08-4a60-9aed-197c11abcb20-kube-api-access-ltvg2\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:08:34.821749 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:34.821744    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bd15740-ee08-4a60-9aed-197c11abcb20-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:08:35.483847 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:35.483754    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4" event={"ID":"7bd15740-ee08-4a60-9aed-197c11abcb20","Type":"ContainerDied","Data":"58aca24675b28c6bff30c72bf2391bdbd1acc0382c6680573285962f298f4d30"}
Jul 09 20:08:35.483847 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:35.483784    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"
Jul 09 20:08:35.483847 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:35.483834    2527 scope.go:117] "RemoveContainer" containerID="afd940c23ff23ad3fd3b17d25cd200d024b0d13dcec8b65523338258fafacaf7"
Jul 09 20:08:35.500173 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:35.500146    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"]
Jul 09 20:08:35.504216 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:35.504192    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-e2f7b0b4-6cdcdf47c4-vxlw4"]
Jul 09 20:08:37.083182 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:08:37.083140    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bd15740-ee08-4a60-9aed-197c11abcb20" path="/var/lib/kubelet/pods/7bd15740-ee08-4a60-9aed-197c11abcb20/volumes"
Jul 09 20:10:19.635300 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635264    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"]
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635710    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd15740-ee08-4a60-9aed-197c11abcb20" containerName="maas-api"
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635724    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd15740-ee08-4a60-9aed-197c11abcb20" containerName="maas-api"
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635732    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5bf184-da9a-411f-acd1-9f5668e88995" containerName="revoke-keys"
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635738    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5bf184-da9a-411f-acd1-9f5668e88995" containerName="revoke-keys"
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635752    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635759    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a461a6-59c3-4909-82e2-7157d7b73484" containerName="revoke-keys"
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635836    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd15740-ee08-4a60-9aed-197c11abcb20" containerName="maas-api"
Jul 09 20:10:19.637622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.635849    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a5bf184-da9a-411f-acd1-9f5668e88995" containerName="revoke-keys"
Jul 09 20:10:19.638819 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.638802    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.641589 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.641561    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9bad9204-serving-cert\""
Jul 09 20:10:19.641765 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.641576    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:10:19.641765 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.641574    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:10:19.641765 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.641609    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:10:19.649011 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.648989    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"]
Jul 09 20:10:19.749196 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.749159    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28c20e4-f566-413a-9bad-02b435e0c4d6-maas-api-tls\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.749196 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.749202    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28c20e4-f566-413a-9bad-02b435e0c4d6-openshift-service-ca\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.749410 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.749294    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkvkf\" (UniqueName: \"kubernetes.io/projected/c28c20e4-f566-413a-9bad-02b435e0c4d6-kube-api-access-tkvkf\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.850730 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.850698    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28c20e4-f566-413a-9bad-02b435e0c4d6-maas-api-tls\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.850944 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.850738    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28c20e4-f566-413a-9bad-02b435e0c4d6-openshift-service-ca\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.850944 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.850872    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkvkf\" (UniqueName: \"kubernetes.io/projected/c28c20e4-f566-413a-9bad-02b435e0c4d6-kube-api-access-tkvkf\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.851348 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.851330    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28c20e4-f566-413a-9bad-02b435e0c4d6-openshift-service-ca\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.853513 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.853486    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28c20e4-f566-413a-9bad-02b435e0c4d6-maas-api-tls\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.859208 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.859189    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkvkf\" (UniqueName: \"kubernetes.io/projected/c28c20e4-f566-413a-9bad-02b435e0c4d6-kube-api-access-tkvkf\") pod \"maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:19.950974 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:19.950889    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:20.087481 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:20.087381    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"]
Jul 09 20:10:20.090724 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:10:20.090696    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28c20e4_f566_413a_9bad_02b435e0c4d6.slice/crio-6158a88dae903bb4db4ad2ce4d2b3ff2319f7bbf4aa54cb42e26be09ce7369af WatchSource:0}: Error finding container 6158a88dae903bb4db4ad2ce4d2b3ff2319f7bbf4aa54cb42e26be09ce7369af: Status 404 returned error can't find the container with id 6158a88dae903bb4db4ad2ce4d2b3ff2319f7bbf4aa54cb42e26be09ce7369af
Jul 09 20:10:20.092528 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:20.092511    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 20:10:21.016404 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:21.016369    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k" event={"ID":"c28c20e4-f566-413a-9bad-02b435e0c4d6","Type":"ContainerStarted","Data":"62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39"}
Jul 09 20:10:21.016848 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:21.016445    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k" event={"ID":"c28c20e4-f566-413a-9bad-02b435e0c4d6","Type":"ContainerStarted","Data":"6158a88dae903bb4db4ad2ce4d2b3ff2319f7bbf4aa54cb42e26be09ce7369af"}
Jul 09 20:10:21.016848 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:21.016484    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:21.032778 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:21.032717    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k" podStartSLOduration=1.637445134 podStartE2EDuration="2.032699105s" totalImagesPullingTime="395.253971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:10:21.032391412 +0000 UTC m=+1832.433526223" watchObservedRunningTime="2026-07-09 20:10:21.032699105 +0000 UTC m=+1832.433833914"
Jul 09 20:10:27.026795 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:27.026768    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:31.190566 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.190524    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"]
Jul 09 20:10:31.194173 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.194157    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"
Jul 09 20:10:31.196835 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.196814    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:10:31.201019 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.200994    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"]
Jul 09 20:10:31.264509 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.264471    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38c9f55-b604-4d35-8fdd-0159f0297d49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd\" (UID: \"f38c9f55-b604-4d35-8fdd-0159f0297d49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"
Jul 09 20:10:31.365372 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.365331    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38c9f55-b604-4d35-8fdd-0159f0297d49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd\" (UID: \"f38c9f55-b604-4d35-8fdd-0159f0297d49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"
Jul 09 20:10:31.366117 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.366096    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38c9f55-b604-4d35-8fdd-0159f0297d49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd\" (UID: \"f38c9f55-b604-4d35-8fdd-0159f0297d49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"
Jul 09 20:10:31.506698 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.506586    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"
Jul 09 20:10:31.641106 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:31.641082    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"]
Jul 09 20:10:31.643263 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:10:31.643226    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38c9f55_b604_4d35_8fdd_0159f0297d49.slice/crio-43e405e9bd5f639223d2d1d29156816b00b53958c476871324d696dd9ba5d055 WatchSource:0}: Error finding container 43e405e9bd5f639223d2d1d29156816b00b53958c476871324d696dd9ba5d055: Status 404 returned error can't find the container with id 43e405e9bd5f639223d2d1d29156816b00b53958c476871324d696dd9ba5d055
Jul 09 20:10:32.072827 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:32.072793    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd" event={"ID":"f38c9f55-b604-4d35-8fdd-0159f0297d49","Type":"ContainerStarted","Data":"99e78f3a44f5cf5c4117755f6cc03288c837c61bff40f003d95603c002e53699"}
Jul 09 20:10:32.072827 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:32.072840    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd" event={"ID":"f38c9f55-b604-4d35-8fdd-0159f0297d49","Type":"ContainerStarted","Data":"43e405e9bd5f639223d2d1d29156816b00b53958c476871324d696dd9ba5d055"}
Jul 09 20:10:32.087990 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:32.087934    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd" podStartSLOduration=1.087914251 podStartE2EDuration="1.087914251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:10:32.086781395 +0000 UTC m=+1843.487916214" watchObservedRunningTime="2026-07-09 20:10:32.087914251 +0000 UTC m=+1843.489049063"
Jul 09 20:10:33.078812 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:33.078781    2527 generic.go:356] "Generic (PLEG): container finished" podID="f38c9f55-b604-4d35-8fdd-0159f0297d49" containerID="99e78f3a44f5cf5c4117755f6cc03288c837c61bff40f003d95603c002e53699" exitCode=0
Jul 09 20:10:33.079225 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:33.078871    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd" event={"ID":"f38c9f55-b604-4d35-8fdd-0159f0297d49","Type":"ContainerDied","Data":"99e78f3a44f5cf5c4117755f6cc03288c837c61bff40f003d95603c002e53699"}
Jul 09 20:10:34.222501 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:34.222474    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"
Jul 09 20:10:34.299909 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:34.299874    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38c9f55-b604-4d35-8fdd-0159f0297d49-maas-api-service-ca\") pod \"f38c9f55-b604-4d35-8fdd-0159f0297d49\" (UID: \"f38c9f55-b604-4d35-8fdd-0159f0297d49\") "
Jul 09 20:10:34.300265 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:34.300242    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f38c9f55-b604-4d35-8fdd-0159f0297d49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f38c9f55-b604-4d35-8fdd-0159f0297d49" (UID: "f38c9f55-b604-4d35-8fdd-0159f0297d49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:10:34.400757 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:34.400672    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38c9f55-b604-4d35-8fdd-0159f0297d49-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:10:35.090133 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:35.090107    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd"
Jul 09 20:10:35.090275 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:35.090144    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd" event={"ID":"f38c9f55-b604-4d35-8fdd-0159f0297d49","Type":"ContainerDied","Data":"43e405e9bd5f639223d2d1d29156816b00b53958c476871324d696dd9ba5d055"}
Jul 09 20:10:35.090275 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:35.090192    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43e405e9bd5f639223d2d1d29156816b00b53958c476871324d696dd9ba5d055"
Jul 09 20:10:41.542860 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.542823    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"]
Jul 09 20:10:41.543368 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.543027    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k" podUID="c28c20e4-f566-413a-9bad-02b435e0c4d6" containerName="maas-api" containerID="cri-o://62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39" gracePeriod=30
Jul 09 20:10:41.794494 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.794436    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:41.873325 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.873275    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28c20e4-f566-413a-9bad-02b435e0c4d6-maas-api-tls\") pod \"c28c20e4-f566-413a-9bad-02b435e0c4d6\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") "
Jul 09 20:10:41.873503 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.873347    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28c20e4-f566-413a-9bad-02b435e0c4d6-openshift-service-ca\") pod \"c28c20e4-f566-413a-9bad-02b435e0c4d6\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") "
Jul 09 20:10:41.873503 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.873369    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkvkf\" (UniqueName: \"kubernetes.io/projected/c28c20e4-f566-413a-9bad-02b435e0c4d6-kube-api-access-tkvkf\") pod \"c28c20e4-f566-413a-9bad-02b435e0c4d6\" (UID: \"c28c20e4-f566-413a-9bad-02b435e0c4d6\") "
Jul 09 20:10:41.873768 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.873737    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c28c20e4-f566-413a-9bad-02b435e0c4d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c28c20e4-f566-413a-9bad-02b435e0c4d6" (UID: "c28c20e4-f566-413a-9bad-02b435e0c4d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:10:41.875535 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.875512    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c28c20e4-f566-413a-9bad-02b435e0c4d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c28c20e4-f566-413a-9bad-02b435e0c4d6" (UID: "c28c20e4-f566-413a-9bad-02b435e0c4d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:10:41.875613 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.875542    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c28c20e4-f566-413a-9bad-02b435e0c4d6-kube-api-access-tkvkf" (OuterVolumeSpecName: "kube-api-access-tkvkf") pod "c28c20e4-f566-413a-9bad-02b435e0c4d6" (UID: "c28c20e4-f566-413a-9bad-02b435e0c4d6"). InnerVolumeSpecName "kube-api-access-tkvkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:10:41.974975 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.974930    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28c20e4-f566-413a-9bad-02b435e0c4d6-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:10:41.974975 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.974967    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28c20e4-f566-413a-9bad-02b435e0c4d6-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:10:41.974975 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:41.974977    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkvkf\" (UniqueName: \"kubernetes.io/projected/c28c20e4-f566-413a-9bad-02b435e0c4d6-kube-api-access-tkvkf\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:10:42.129146 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.129046    2527 generic.go:356] "Generic (PLEG): container finished" podID="c28c20e4-f566-413a-9bad-02b435e0c4d6" containerID="62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39" exitCode=0
Jul 09 20:10:42.129146 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.129088    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"
Jul 09 20:10:42.129146 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.129108    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k" event={"ID":"c28c20e4-f566-413a-9bad-02b435e0c4d6","Type":"ContainerDied","Data":"62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39"}
Jul 09 20:10:42.129415 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.129157    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k" event={"ID":"c28c20e4-f566-413a-9bad-02b435e0c4d6","Type":"ContainerDied","Data":"6158a88dae903bb4db4ad2ce4d2b3ff2319f7bbf4aa54cb42e26be09ce7369af"}
Jul 09 20:10:42.129415 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.129180    2527 scope.go:117] "RemoveContainer" containerID="62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39"
Jul 09 20:10:42.138533 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.138508    2527 scope.go:117] "RemoveContainer" containerID="62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39"
Jul 09 20:10:42.138818 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:10:42.138798    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39\": container with ID starting with 62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39 not found: ID does not exist" containerID="62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39"
Jul 09 20:10:42.138874 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.138826    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39"} err="failed to get container status \"62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39\": rpc error: code = NotFound desc = could not find container \"62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39\": container with ID starting with 62edf3e5214642811ff06e40cdd05accfac159f5d7892b128a3f968ec6167e39 not found: ID does not exist"
Jul 09 20:10:42.152338 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.152305    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"]
Jul 09 20:10:42.154555 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:42.154533    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9bad9204-5b795b5c59-fq75k"]
Jul 09 20:10:43.077484 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:10:43.077454    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c28c20e4-f566-413a-9bad-02b435e0c4d6" path="/var/lib/kubelet/pods/c28c20e4-f566-413a-9bad-02b435e0c4d6/volumes"
Jul 09 20:11:19.491583 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.491547    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"]
Jul 09 20:11:19.492189 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.492009    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c28c20e4-f566-413a-9bad-02b435e0c4d6" containerName="maas-api"
Jul 09 20:11:19.492189 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.492022    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="c28c20e4-f566-413a-9bad-02b435e0c4d6" containerName="maas-api"
Jul 09 20:11:19.492189 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.492042    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38c9f55-b604-4d35-8fdd-0159f0297d49" containerName="revoke-keys"
Jul 09 20:11:19.492189 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.492048    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38c9f55-b604-4d35-8fdd-0159f0297d49" containerName="revoke-keys"
Jul 09 20:11:19.492189 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.492122    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38c9f55-b604-4d35-8fdd-0159f0297d49" containerName="revoke-keys"
Jul 09 20:11:19.492189 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.492131    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="c28c20e4-f566-413a-9bad-02b435e0c4d6" containerName="maas-api"
Jul 09 20:11:19.495309 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.495288    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.498006 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.497982    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-71d42f8b-serving-cert\""
Jul 09 20:11:19.498259 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.498240    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:11:19.498311 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.498245    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:11:19.499272 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.499251    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:11:19.509329 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.509302    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"]
Jul 09 20:11:19.545988 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.545956    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g9f6\" (UniqueName: \"kubernetes.io/projected/bccdf6e8-328e-48c6-9e80-9768237534b6-kube-api-access-6g9f6\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.546187 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.546027    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccdf6e8-328e-48c6-9e80-9768237534b6-maas-api-tls\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.546187 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.546073    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccdf6e8-328e-48c6-9e80-9768237534b6-openshift-service-ca\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.647382 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.647345    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g9f6\" (UniqueName: \"kubernetes.io/projected/bccdf6e8-328e-48c6-9e80-9768237534b6-kube-api-access-6g9f6\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.647579 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.647408    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccdf6e8-328e-48c6-9e80-9768237534b6-maas-api-tls\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.647579 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.647432    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccdf6e8-328e-48c6-9e80-9768237534b6-openshift-service-ca\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.648060 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.648040    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccdf6e8-328e-48c6-9e80-9768237534b6-openshift-service-ca\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.650070 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.650050    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccdf6e8-328e-48c6-9e80-9768237534b6-maas-api-tls\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.655118 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.655093    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g9f6\" (UniqueName: \"kubernetes.io/projected/bccdf6e8-328e-48c6-9e80-9768237534b6-kube-api-access-6g9f6\") pod \"maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.806375 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.806341    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:19.943681 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:19.943636    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"]
Jul 09 20:11:19.945328 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:11:19.945301    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccdf6e8_328e_48c6_9e80_9768237534b6.slice/crio-d17bc869644e3c9432eeb3b3d0f4d5f8fc352b86a29de9e2603fe9f18d03ebf9 WatchSource:0}: Error finding container d17bc869644e3c9432eeb3b3d0f4d5f8fc352b86a29de9e2603fe9f18d03ebf9: Status 404 returned error can't find the container with id d17bc869644e3c9432eeb3b3d0f4d5f8fc352b86a29de9e2603fe9f18d03ebf9
Jul 09 20:11:20.319177 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:20.319142    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w" event={"ID":"bccdf6e8-328e-48c6-9e80-9768237534b6","Type":"ContainerStarted","Data":"d17bc869644e3c9432eeb3b3d0f4d5f8fc352b86a29de9e2603fe9f18d03ebf9"}
Jul 09 20:11:21.325523 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:21.325476    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w" event={"ID":"bccdf6e8-328e-48c6-9e80-9768237534b6","Type":"ContainerStarted","Data":"36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a"}
Jul 09 20:11:21.325942 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:21.325635    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:21.345326 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:21.345272    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w" podStartSLOduration=1.899739804 podStartE2EDuration="2.345256537s" totalImagesPullingTime="445.516733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:11:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:11:21.342894491 +0000 UTC m=+1892.744029301" watchObservedRunningTime="2026-07-09 20:11:21.345256537 +0000 UTC m=+1892.746391358"
Jul 09 20:11:27.337936 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:27.337905    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:11:36.294795 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.294758    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"]
Jul 09 20:11:36.298564 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.298531    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.301475 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.301453    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-71d42f8b\"/\"openshift-service-ca.crt\""
Jul 09 20:11:36.301607 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.301554    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-71d42f8b\"/\"default-dockercfg-7f5pk\""
Jul 09 20:11:36.301717 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.301453    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-71d42f8b\"/\"kube-root-ca.crt\""
Jul 09 20:11:36.302822 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.302791    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-71d42f8b\"/\"e2e-lifecycle-model-71d42f8b-kserve-self-signed-certs\""
Jul 09 20:11:36.307173 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.307148    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"]
Jul 09 20:11:36.421423 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.421390    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-model-cache\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.421423 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.421429    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-home\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.421663 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.421529    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88sc9\" (UniqueName: \"kubernetes.io/projected/7057c63e-f7bb-4c6e-b274-25e107e861f2-kube-api-access-88sc9\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.421663 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.421559    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-tmp-dir\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.421663 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.421579    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-dshm\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.421663 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.421605    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7057c63e-f7bb-4c6e-b274-25e107e861f2-tls-certs\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.522851 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.522805    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-model-cache\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.522851 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.522859    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-home\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.523154 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.522929    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88sc9\" (UniqueName: \"kubernetes.io/projected/7057c63e-f7bb-4c6e-b274-25e107e861f2-kube-api-access-88sc9\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.523154 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.522968    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-tmp-dir\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.523154 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.522997    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-dshm\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.523154 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.523056    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7057c63e-f7bb-4c6e-b274-25e107e861f2-tls-certs\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.523355 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.523300    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-model-cache\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.523355 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.523314    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-home\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.523462 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.523377    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-tmp-dir\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.525477 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.525450    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-dshm\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.525618 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.525600    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7057c63e-f7bb-4c6e-b274-25e107e861f2-tls-certs\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.530930 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.530910    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88sc9\" (UniqueName: \"kubernetes.io/projected/7057c63e-f7bb-4c6e-b274-25e107e861f2-kube-api-access-88sc9\") pod \"e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") " pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.611941 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.611848    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:36.754709 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:36.754677    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"]
Jul 09 20:11:36.756761 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:11:36.756735    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7057c63e_f7bb_4c6e_b274_25e107e861f2.slice/crio-5aa02bebfe6c4e36a994400411d067f55e320cee3a85719e781a08b1c1aab144 WatchSource:0}: Error finding container 5aa02bebfe6c4e36a994400411d067f55e320cee3a85719e781a08b1c1aab144: Status 404 returned error can't find the container with id 5aa02bebfe6c4e36a994400411d067f55e320cee3a85719e781a08b1c1aab144
Jul 09 20:11:37.416905 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:37.416864    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv" event={"ID":"7057c63e-f7bb-4c6e-b274-25e107e861f2","Type":"ContainerStarted","Data":"627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295"}
Jul 09 20:11:37.416905 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:37.416908    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv" event={"ID":"7057c63e-f7bb-4c6e-b274-25e107e861f2","Type":"ContainerStarted","Data":"5aa02bebfe6c4e36a994400411d067f55e320cee3a85719e781a08b1c1aab144"}
Jul 09 20:11:37.436666 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:37.435607    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv" podStartSLOduration=1.435577967 podStartE2EDuration="1.435577967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:11:37.432256662 +0000 UTC m=+1908.833391473" watchObservedRunningTime="2026-07-09 20:11:37.435577967 +0000 UTC m=+1908.836712778"
Jul 09 20:11:46.613067 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:46.613029    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:46.613539 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:46.613090    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:46.625768 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:46.625737    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:11:47.482150 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:11:47.482112    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:12:12.964623 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:12.964576    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"]
Jul 09 20:12:12.965230 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:12.964844    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w" podUID="bccdf6e8-328e-48c6-9e80-9768237534b6" containerName="maas-api" containerID="cri-o://36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a" gracePeriod=30
Jul 09 20:12:13.214094 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.214071    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:12:13.393727 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.393682    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccdf6e8-328e-48c6-9e80-9768237534b6-openshift-service-ca\") pod \"bccdf6e8-328e-48c6-9e80-9768237534b6\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") "
Jul 09 20:12:13.393727 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.393732    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g9f6\" (UniqueName: \"kubernetes.io/projected/bccdf6e8-328e-48c6-9e80-9768237534b6-kube-api-access-6g9f6\") pod \"bccdf6e8-328e-48c6-9e80-9768237534b6\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") "
Jul 09 20:12:13.393992 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.393833    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccdf6e8-328e-48c6-9e80-9768237534b6-maas-api-tls\") pod \"bccdf6e8-328e-48c6-9e80-9768237534b6\" (UID: \"bccdf6e8-328e-48c6-9e80-9768237534b6\") "
Jul 09 20:12:13.394155 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.394128    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bccdf6e8-328e-48c6-9e80-9768237534b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bccdf6e8-328e-48c6-9e80-9768237534b6" (UID: "bccdf6e8-328e-48c6-9e80-9768237534b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:12:13.396079 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.396051    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bccdf6e8-328e-48c6-9e80-9768237534b6-kube-api-access-6g9f6" (OuterVolumeSpecName: "kube-api-access-6g9f6") pod "bccdf6e8-328e-48c6-9e80-9768237534b6" (UID: "bccdf6e8-328e-48c6-9e80-9768237534b6"). InnerVolumeSpecName "kube-api-access-6g9f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:12:13.396179 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.396073    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bccdf6e8-328e-48c6-9e80-9768237534b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bccdf6e8-328e-48c6-9e80-9768237534b6" (UID: "bccdf6e8-328e-48c6-9e80-9768237534b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:12:13.495291 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.495245    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccdf6e8-328e-48c6-9e80-9768237534b6-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:13.495291 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.495288    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g9f6\" (UniqueName: \"kubernetes.io/projected/bccdf6e8-328e-48c6-9e80-9768237534b6-kube-api-access-6g9f6\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:13.495497 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.495304    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccdf6e8-328e-48c6-9e80-9768237534b6-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:13.608351 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.608315    2527 generic.go:356] "Generic (PLEG): container finished" podID="bccdf6e8-328e-48c6-9e80-9768237534b6" containerID="36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a" exitCode=0
Jul 09 20:12:13.608537 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.608381    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"
Jul 09 20:12:13.608537 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.608401    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w" event={"ID":"bccdf6e8-328e-48c6-9e80-9768237534b6","Type":"ContainerDied","Data":"36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a"}
Jul 09 20:12:13.608537 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.608482    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w" event={"ID":"bccdf6e8-328e-48c6-9e80-9768237534b6","Type":"ContainerDied","Data":"d17bc869644e3c9432eeb3b3d0f4d5f8fc352b86a29de9e2603fe9f18d03ebf9"}
Jul 09 20:12:13.608537 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.608505    2527 scope.go:117] "RemoveContainer" containerID="36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a"
Jul 09 20:12:13.617903 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.617882    2527 scope.go:117] "RemoveContainer" containerID="36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a"
Jul 09 20:12:13.618171 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:12:13.618150    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a\": container with ID starting with 36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a not found: ID does not exist" containerID="36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a"
Jul 09 20:12:13.618239 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.618179    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a"} err="failed to get container status \"36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a\": rpc error: code = NotFound desc = could not find container \"36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a\": container with ID starting with 36c0c384d3feff10601252a754a71e3e8317925f03293cb4f77916ca40d7261a not found: ID does not exist"
Jul 09 20:12:13.628869 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.628846    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"]
Jul 09 20:12:13.632504 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:13.632478    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-71d42f8b-676f6d9598-pbh7w"]
Jul 09 20:12:15.077217 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:15.077183    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bccdf6e8-328e-48c6-9e80-9768237534b6" path="/var/lib/kubelet/pods/bccdf6e8-328e-48c6-9e80-9768237534b6/volumes"
Jul 09 20:12:22.655488 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:22.655443    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"]
Jul 09 20:12:37.677121 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:37.677055    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv" podUID="7057c63e-f7bb-4c6e-b274-25e107e861f2" containerName="main" containerID="cri-o://627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295" gracePeriod=45
Jul 09 20:12:37.933175 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:37.933113    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:12:38.035982 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.035951    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-home\") pod \"7057c63e-f7bb-4c6e-b274-25e107e861f2\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") "
Jul 09 20:12:38.036158 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036023    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7057c63e-f7bb-4c6e-b274-25e107e861f2-tls-certs\") pod \"7057c63e-f7bb-4c6e-b274-25e107e861f2\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") "
Jul 09 20:12:38.036158 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036045    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-dshm\") pod \"7057c63e-f7bb-4c6e-b274-25e107e861f2\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") "
Jul 09 20:12:38.036158 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036112    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-model-cache\") pod \"7057c63e-f7bb-4c6e-b274-25e107e861f2\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") "
Jul 09 20:12:38.036158 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036135    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-tmp-dir\") pod \"7057c63e-f7bb-4c6e-b274-25e107e861f2\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") "
Jul 09 20:12:38.036361 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036164    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88sc9\" (UniqueName: \"kubernetes.io/projected/7057c63e-f7bb-4c6e-b274-25e107e861f2-kube-api-access-88sc9\") pod \"7057c63e-f7bb-4c6e-b274-25e107e861f2\" (UID: \"7057c63e-f7bb-4c6e-b274-25e107e861f2\") "
Jul 09 20:12:38.036361 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036267    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-home" (OuterVolumeSpecName: "home") pod "7057c63e-f7bb-4c6e-b274-25e107e861f2" (UID: "7057c63e-f7bb-4c6e-b274-25e107e861f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:12:38.036361 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036343    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-model-cache" (OuterVolumeSpecName: "model-cache") pod "7057c63e-f7bb-4c6e-b274-25e107e861f2" (UID: "7057c63e-f7bb-4c6e-b274-25e107e861f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:12:38.036512 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036431    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:38.036512 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036435    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7057c63e-f7bb-4c6e-b274-25e107e861f2" (UID: "7057c63e-f7bb-4c6e-b274-25e107e861f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:12:38.036512 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.036450    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:38.038530 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.038506    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-dshm" (OuterVolumeSpecName: "dshm") pod "7057c63e-f7bb-4c6e-b274-25e107e861f2" (UID: "7057c63e-f7bb-4c6e-b274-25e107e861f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:12:38.038530 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.038521    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7057c63e-f7bb-4c6e-b274-25e107e861f2-kube-api-access-88sc9" (OuterVolumeSpecName: "kube-api-access-88sc9") pod "7057c63e-f7bb-4c6e-b274-25e107e861f2" (UID: "7057c63e-f7bb-4c6e-b274-25e107e861f2"). InnerVolumeSpecName "kube-api-access-88sc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:12:38.038674 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.038560    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7057c63e-f7bb-4c6e-b274-25e107e861f2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7057c63e-f7bb-4c6e-b274-25e107e861f2" (UID: "7057c63e-f7bb-4c6e-b274-25e107e861f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:12:38.137835 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.137773    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:38.137835 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.137810    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88sc9\" (UniqueName: \"kubernetes.io/projected/7057c63e-f7bb-4c6e-b274-25e107e861f2-kube-api-access-88sc9\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:38.137835 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.137825    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7057c63e-f7bb-4c6e-b274-25e107e861f2-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:38.137835 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.137840    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7057c63e-f7bb-4c6e-b274-25e107e861f2-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:12:38.740086 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.740053    2527 generic.go:356] "Generic (PLEG): container finished" podID="7057c63e-f7bb-4c6e-b274-25e107e861f2" containerID="627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295" exitCode=0
Jul 09 20:12:38.740542 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.740095    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv" event={"ID":"7057c63e-f7bb-4c6e-b274-25e107e861f2","Type":"ContainerDied","Data":"627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295"}
Jul 09 20:12:38.740542 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.740100    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"
Jul 09 20:12:38.740542 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.740122    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv" event={"ID":"7057c63e-f7bb-4c6e-b274-25e107e861f2","Type":"ContainerDied","Data":"5aa02bebfe6c4e36a994400411d067f55e320cee3a85719e781a08b1c1aab144"}
Jul 09 20:12:38.740542 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.740142    2527 scope.go:117] "RemoveContainer" containerID="627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295"
Jul 09 20:12:38.749927 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.749910    2527 scope.go:117] "RemoveContainer" containerID="627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295"
Jul 09 20:12:38.750203 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:12:38.750183    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295\": container with ID starting with 627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295 not found: ID does not exist" containerID="627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295"
Jul 09 20:12:38.750242 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.750214    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295"} err="failed to get container status \"627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295\": rpc error: code = NotFound desc = could not find container \"627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295\": container with ID starting with 627cdf13fe074eddda185ebd60da9825b3f956a872a0107667e4b33ea030d295 not found: ID does not exist"
Jul 09 20:12:38.760895 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.760863    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"]
Jul 09 20:12:38.764335 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:38.764309    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-71d42f8b/e2e-lifecycle-model-71d42f8b-kserve-57fd6f4855-q9klv"]
Jul 09 20:12:39.078334 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:12:39.078303    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7057c63e-f7bb-4c6e-b274-25e107e861f2" path="/var/lib/kubelet/pods/7057c63e-f7bb-4c6e-b274-25e107e861f2/volumes"
Jul 09 20:13:49.219363 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:13:49.219329    2527 scope.go:117] "RemoveContainer" containerID="c6d6454e07def1eee09538b7ce93e1a40255c8ba68e538dd4fc5305eea57d423"
Jul 09 20:15:00.184541 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.184509    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb"]
Jul 09 20:15:00.185031 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.184948    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bccdf6e8-328e-48c6-9e80-9768237534b6" containerName="maas-api"
Jul 09 20:15:00.185031 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.184961    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="bccdf6e8-328e-48c6-9e80-9768237534b6" containerName="maas-api"
Jul 09 20:15:00.185031 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.184974    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7057c63e-f7bb-4c6e-b274-25e107e861f2" containerName="main"
Jul 09 20:15:00.185031 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.184979    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="7057c63e-f7bb-4c6e-b274-25e107e861f2" containerName="main"
Jul 09 20:15:00.185162 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.185051    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="7057c63e-f7bb-4c6e-b274-25e107e861f2" containerName="main"
Jul 09 20:15:00.185162 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.185060    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="bccdf6e8-328e-48c6-9e80-9768237534b6" containerName="maas-api"
Jul 09 20:15:00.188230 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.188214    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb"
Jul 09 20:15:00.192086 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.192065    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:15:00.197772 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.197753    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb"
Jul 09 20:15:00.314801 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.314768    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb"]
Jul 09 20:15:00.551023 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:00.550985    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb"]
Jul 09 20:15:00.557619 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:15:00.557584    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90dc504_0d11_4a54_aa2e_04dab08b6a51.slice/crio-b6be9d25eeec168944c14a2ea8a3ec5b0fb2d476f562070e6361da338cfe75c6 WatchSource:0}: Error finding container b6be9d25eeec168944c14a2ea8a3ec5b0fb2d476f562070e6361da338cfe75c6: Status 404 returned error can't find the container with id b6be9d25eeec168944c14a2ea8a3ec5b0fb2d476f562070e6361da338cfe75c6
Jul 09 20:15:01.453490 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:01.453453    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb" event={"ID":"d90dc504-0d11-4a54-aa2e-04dab08b6a51","Type":"ContainerStarted","Data":"584249eefa4987e1d4b2f4dde9d89dfa1a1c0d930a940e8bc3b6c991d072e6bd"}
Jul 09 20:15:01.453900 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:01.453517    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb" event={"ID":"d90dc504-0d11-4a54-aa2e-04dab08b6a51","Type":"ContainerStarted","Data":"b6be9d25eeec168944c14a2ea8a3ec5b0fb2d476f562070e6361da338cfe75c6"}
Jul 09 20:15:02.460426 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:02.460390    2527 generic.go:356] "Generic (PLEG): container finished" podID="d90dc504-0d11-4a54-aa2e-04dab08b6a51" containerID="584249eefa4987e1d4b2f4dde9d89dfa1a1c0d930a940e8bc3b6c991d072e6bd" exitCode=0
Jul 09 20:15:02.460824 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:02.460436    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb" event={"ID":"d90dc504-0d11-4a54-aa2e-04dab08b6a51","Type":"ContainerDied","Data":"584249eefa4987e1d4b2f4dde9d89dfa1a1c0d930a940e8bc3b6c991d072e6bd"}
Jul 09 20:15:03.616011 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:03.615987    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb"
Jul 09 20:15:04.470851 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:04.470821    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb"
Jul 09 20:15:04.471039 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:04.470856    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727135-9q2mb" event={"ID":"d90dc504-0d11-4a54-aa2e-04dab08b6a51","Type":"ContainerDied","Data":"b6be9d25eeec168944c14a2ea8a3ec5b0fb2d476f562070e6361da338cfe75c6"}
Jul 09 20:15:04.471039 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:15:04.470912    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6be9d25eeec168944c14a2ea8a3ec5b0fb2d476f562070e6361da338cfe75c6"
Jul 09 20:17:27.631430 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.631401    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"]
Jul 09 20:17:27.632059 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.631885    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d90dc504-0d11-4a54-aa2e-04dab08b6a51" containerName="cleanup"
Jul 09 20:17:27.632059 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.631897    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d90dc504-0d11-4a54-aa2e-04dab08b6a51" containerName="cleanup"
Jul 09 20:17:27.632059 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.631985    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d90dc504-0d11-4a54-aa2e-04dab08b6a51" containerName="cleanup"
Jul 09 20:17:27.643525 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.643485    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.646485 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.646460    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-35f9ad-serving-cert\""
Jul 09 20:17:27.646638 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.646526    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:17:27.646638 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.646562    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:17:27.646810 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.646789    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"]
Jul 09 20:17:27.647546 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.647529    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:17:27.781497 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.781455    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8422e67a-dd38-4507-aa72-d20aa8eb847b-maas-api-tls\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.781701 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.781592    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62chd\" (UniqueName: \"kubernetes.io/projected/8422e67a-dd38-4507-aa72-d20aa8eb847b-kube-api-access-62chd\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.781701 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.781694    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8422e67a-dd38-4507-aa72-d20aa8eb847b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.882570 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.882477    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8422e67a-dd38-4507-aa72-d20aa8eb847b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.882570 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.882527    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8422e67a-dd38-4507-aa72-d20aa8eb847b-maas-api-tls\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.882830 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.882598    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62chd\" (UniqueName: \"kubernetes.io/projected/8422e67a-dd38-4507-aa72-d20aa8eb847b-kube-api-access-62chd\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.883330 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.883301    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8422e67a-dd38-4507-aa72-d20aa8eb847b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.885353 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.885331    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8422e67a-dd38-4507-aa72-d20aa8eb847b-maas-api-tls\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.891203 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.891181    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62chd\" (UniqueName: \"kubernetes.io/projected/8422e67a-dd38-4507-aa72-d20aa8eb847b-kube-api-access-62chd\") pod \"maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:27.956459 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:27.956418    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:28.097600 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:28.097576    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"]
Jul 09 20:17:28.099779 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:17:28.099749    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8422e67a_dd38_4507_aa72_d20aa8eb847b.slice/crio-c8e1e9504e37fd480ca7889f4c4a55a248233d9b984754401fcac13fd6a27366 WatchSource:0}: Error finding container c8e1e9504e37fd480ca7889f4c4a55a248233d9b984754401fcac13fd6a27366: Status 404 returned error can't find the container with id c8e1e9504e37fd480ca7889f4c4a55a248233d9b984754401fcac13fd6a27366
Jul 09 20:17:28.101500 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:28.101483    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 20:17:28.208965 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:28.208860    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b" event={"ID":"8422e67a-dd38-4507-aa72-d20aa8eb847b","Type":"ContainerStarted","Data":"c8e1e9504e37fd480ca7889f4c4a55a248233d9b984754401fcac13fd6a27366"}
Jul 09 20:17:29.215308 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:29.215275    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b" event={"ID":"8422e67a-dd38-4507-aa72-d20aa8eb847b","Type":"ContainerStarted","Data":"4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d"}
Jul 09 20:17:29.215697 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:29.215381    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:29.233930 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:29.233880    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b" podStartSLOduration=1.781716488 podStartE2EDuration="2.23386274s" totalImagesPullingTime="452.146252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:17:29.231454885 +0000 UTC m=+2260.632589696" watchObservedRunningTime="2026-07-09 20:17:29.23386274 +0000 UTC m=+2260.634997550"
Jul 09 20:17:35.225281 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.225249    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:17:35.711878 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.711840    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"]
Jul 09 20:17:35.716374 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.716352    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.719196 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.719166    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-54be63-serving-cert\""
Jul 09 20:17:35.727133 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.727102    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"]
Jul 09 20:17:35.866678 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.866605    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.866864 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.866696    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpvpq\" (UniqueName: \"kubernetes.io/projected/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-kube-api-access-rpvpq\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.866864 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.866748    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-maas-api-tls\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.968177 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.968080    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.968177 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.968132    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpvpq\" (UniqueName: \"kubernetes.io/projected/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-kube-api-access-rpvpq\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.968177 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.968169    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-maas-api-tls\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.968781 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.968760    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.970710 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.970693    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-maas-api-tls\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:35.977372 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:35.977348    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpvpq\" (UniqueName: \"kubernetes.io/projected/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-kube-api-access-rpvpq\") pod \"maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:36.029135 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:36.029098    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:36.172181 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:36.172158    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"]
Jul 09 20:17:36.175005 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:17:36.174977    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d2faf14_afd8_4253_8cf4_f5ec6b598df3.slice/crio-9c58ab2242abbd7ae4e1e70ce42ed9a80e2a992654a7f37a8e0501ecb84ed815 WatchSource:0}: Error finding container 9c58ab2242abbd7ae4e1e70ce42ed9a80e2a992654a7f37a8e0501ecb84ed815: Status 404 returned error can't find the container with id 9c58ab2242abbd7ae4e1e70ce42ed9a80e2a992654a7f37a8e0501ecb84ed815
Jul 09 20:17:36.252194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:36.252106    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2" event={"ID":"4d2faf14-afd8-4253-8cf4-f5ec6b598df3","Type":"ContainerStarted","Data":"9c58ab2242abbd7ae4e1e70ce42ed9a80e2a992654a7f37a8e0501ecb84ed815"}
Jul 09 20:17:37.258841 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:37.258802    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2" event={"ID":"4d2faf14-afd8-4253-8cf4-f5ec6b598df3","Type":"ContainerStarted","Data":"3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae"}
Jul 09 20:17:37.259242 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:37.258884    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:37.275318 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:37.275259    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2" podStartSLOduration=1.88659707 podStartE2EDuration="2.27523906s" totalImagesPullingTime="388.64199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:17:37.274628998 +0000 UTC m=+2268.675763808" watchObservedRunningTime="2026-07-09 20:17:37.27523906 +0000 UTC m=+2268.676373871"
Jul 09 20:17:43.269955 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:43.269929    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:17:52.285338 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.285302    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"]
Jul 09 20:17:52.289279 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.289263    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.293119 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.293080    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-35f9ad\"/\"kube-root-ca.crt\""
Jul 09 20:17:52.293119 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.293088    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-35f9ad\"/\"default-dockercfg-8sgrf\""
Jul 09 20:17:52.293303 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.293149    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-35f9ad\"/\"test-model-35f9ad-kserve-self-signed-certs\""
Jul 09 20:17:52.293303 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.293115    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-35f9ad\"/\"openshift-service-ca.crt\""
Jul 09 20:17:52.298106 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.298080    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"]
Jul 09 20:17:52.332464 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.332434    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-home\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.332724 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.332493    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr9tj\" (UniqueName: \"kubernetes.io/projected/b0157033-c319-4cab-a8d3-c42c45869892-kube-api-access-fr9tj\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.332724 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.332539    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-model-cache\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.332724 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.332584    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-dshm\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.332724 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.332620    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0157033-c319-4cab-a8d3-c42c45869892-tls-certs\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.332902 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.332732    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-tmp-dir\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.433472 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.433438    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr9tj\" (UniqueName: \"kubernetes.io/projected/b0157033-c319-4cab-a8d3-c42c45869892-kube-api-access-fr9tj\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.433472 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.433476    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-model-cache\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.433696 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.433518    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-dshm\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.433696 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.433555    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0157033-c319-4cab-a8d3-c42c45869892-tls-certs\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.433696 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.433618    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-tmp-dir\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.433863 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.433786    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-home\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.434040 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.434020    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-model-cache\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.434177 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.434102    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-tmp-dir\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.434231 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.434199    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-home\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.436007 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.435979    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-dshm\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.436268 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.436251    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0157033-c319-4cab-a8d3-c42c45869892-tls-certs\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.442044 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.442015    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr9tj\" (UniqueName: \"kubernetes.io/projected/b0157033-c319-4cab-a8d3-c42c45869892-kube-api-access-fr9tj\") pod \"test-model-35f9ad-kserve-6dccbb9c97-dcj66\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") " pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.600257 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.600161    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:17:52.744774 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:52.744747    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"]
Jul 09 20:17:52.746534 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:17:52.746508    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0157033_c319_4cab_a8d3_c42c45869892.slice/crio-c488edc499e6f75f576b24a216ea73eea4ea9c3496f350fc443237402b701385 WatchSource:0}: Error finding container c488edc499e6f75f576b24a216ea73eea4ea9c3496f350fc443237402b701385: Status 404 returned error can't find the container with id c488edc499e6f75f576b24a216ea73eea4ea9c3496f350fc443237402b701385
Jul 09 20:17:53.346035 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:53.345998    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66" event={"ID":"b0157033-c319-4cab-a8d3-c42c45869892","Type":"ContainerStarted","Data":"6f464668962799e6bfe7da5356a084941c2dfb8bead0f3a803dc32d57871e4ed"}
Jul 09 20:17:53.346035 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:53.346040    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66" event={"ID":"b0157033-c319-4cab-a8d3-c42c45869892","Type":"ContainerStarted","Data":"c488edc499e6f75f576b24a216ea73eea4ea9c3496f350fc443237402b701385"}
Jul 09 20:17:53.363792 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:17:53.363737    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66" podStartSLOduration=1.363718667 podStartE2EDuration="1.363718667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:17:53.36199566 +0000 UTC m=+2284.763130473" watchObservedRunningTime="2026-07-09 20:17:53.363718667 +0000 UTC m=+2284.764853476"
Jul 09 20:18:02.600439 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:02.600400    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:18:02.601090 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:02.600461    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:18:02.612938 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:02.612913    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:18:03.409631 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:03.409602    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:18:31.280985 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.280952    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"]
Jul 09 20:18:31.284996 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.284979    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.289024 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.288993    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-54be63\"/\"test-model-54be63-kserve-self-signed-certs\""
Jul 09 20:18:31.289024 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.289022    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-54be63\"/\"kube-root-ca.crt\""
Jul 09 20:18:31.289211 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.289028    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-54be63\"/\"openshift-service-ca.crt\""
Jul 09 20:18:31.289211 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.289035    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-54be63\"/\"default-dockercfg-g7m85\""
Jul 09 20:18:31.293237 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.293118    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"]
Jul 09 20:18:31.324506 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.324471    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-model-cache\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.324712 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.324524    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cca4abc6-6662-44ad-9850-9c492e955ee7-tls-certs\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.324712 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.324545    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-tmp-dir\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.324712 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.324596    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-dshm\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.324712 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.324627    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-home\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.324712 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.324676    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mbxf\" (UniqueName: \"kubernetes.io/projected/cca4abc6-6662-44ad-9850-9c492e955ee7-kube-api-access-8mbxf\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.425873 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.425838    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-model-cache\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426044 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.425888    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cca4abc6-6662-44ad-9850-9c492e955ee7-tls-certs\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426044 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.425937    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-tmp-dir\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426044 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.425993    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-dshm\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426164 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.426042    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-home\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426164 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.426141    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mbxf\" (UniqueName: \"kubernetes.io/projected/cca4abc6-6662-44ad-9850-9c492e955ee7-kube-api-access-8mbxf\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426390 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.426352    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-model-cache\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426390 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.426383    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-tmp-dir\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.426568 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.426552    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-home\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.428484 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.428462    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-dshm\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.428676 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.428637    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cca4abc6-6662-44ad-9850-9c492e955ee7-tls-certs\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.434896 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.434872    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mbxf\" (UniqueName: \"kubernetes.io/projected/cca4abc6-6662-44ad-9850-9c492e955ee7-kube-api-access-8mbxf\") pod \"test-model-54be63-kserve-65f9484dc5-cmcgk\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") " pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.596858 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.596753    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:31.741439 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:31.741408    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"]
Jul 09 20:18:31.743960 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:18:31.743936    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcca4abc6_6662_44ad_9850_9c492e955ee7.slice/crio-932693fa5892b7abf511893458961aa14ead6c2531bb25171aa35ce1da0d5a0a WatchSource:0}: Error finding container 932693fa5892b7abf511893458961aa14ead6c2531bb25171aa35ce1da0d5a0a: Status 404 returned error can't find the container with id 932693fa5892b7abf511893458961aa14ead6c2531bb25171aa35ce1da0d5a0a
Jul 09 20:18:32.551623 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:32.551583    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk" event={"ID":"cca4abc6-6662-44ad-9850-9c492e955ee7","Type":"ContainerStarted","Data":"6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc"}
Jul 09 20:18:32.552094 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:32.551633    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk" event={"ID":"cca4abc6-6662-44ad-9850-9c492e955ee7","Type":"ContainerStarted","Data":"932693fa5892b7abf511893458961aa14ead6c2531bb25171aa35ce1da0d5a0a"}
Jul 09 20:18:32.572874 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:32.572805    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk" podStartSLOduration=1.572783227 podStartE2EDuration="1.572783227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:18:32.56766819 +0000 UTC m=+2323.968802992" watchObservedRunningTime="2026-07-09 20:18:32.572783227 +0000 UTC m=+2323.973918038"
Jul 09 20:18:41.597492 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:41.597456    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:41.597980 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:41.597548    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:41.610871 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:41.610844    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:18:42.621617 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:18:42.621587    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:19:04.568738 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:04.568701    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"]
Jul 09 20:19:05.493813 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:05.493777    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"]
Jul 09 20:19:16.087777 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.087738    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"]
Jul 09 20:19:16.088406 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.088017    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b" podUID="8422e67a-dd38-4507-aa72-d20aa8eb847b" containerName="maas-api" containerID="cri-o://4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d" gracePeriod=30
Jul 09 20:19:16.538205 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.538178    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:19:16.572739 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.572708    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62chd\" (UniqueName: \"kubernetes.io/projected/8422e67a-dd38-4507-aa72-d20aa8eb847b-kube-api-access-62chd\") pod \"8422e67a-dd38-4507-aa72-d20aa8eb847b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") "
Jul 09 20:19:16.572904 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.572771    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8422e67a-dd38-4507-aa72-d20aa8eb847b-openshift-service-ca\") pod \"8422e67a-dd38-4507-aa72-d20aa8eb847b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") "
Jul 09 20:19:16.572989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.572968    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8422e67a-dd38-4507-aa72-d20aa8eb847b-maas-api-tls\") pod \"8422e67a-dd38-4507-aa72-d20aa8eb847b\" (UID: \"8422e67a-dd38-4507-aa72-d20aa8eb847b\") "
Jul 09 20:19:16.573139 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.573120    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8422e67a-dd38-4507-aa72-d20aa8eb847b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8422e67a-dd38-4507-aa72-d20aa8eb847b" (UID: "8422e67a-dd38-4507-aa72-d20aa8eb847b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:19:16.573353 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.573336    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8422e67a-dd38-4507-aa72-d20aa8eb847b-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:16.575048 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.575021    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8422e67a-dd38-4507-aa72-d20aa8eb847b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8422e67a-dd38-4507-aa72-d20aa8eb847b" (UID: "8422e67a-dd38-4507-aa72-d20aa8eb847b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:19:16.575167 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.575147    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8422e67a-dd38-4507-aa72-d20aa8eb847b-kube-api-access-62chd" (OuterVolumeSpecName: "kube-api-access-62chd") pod "8422e67a-dd38-4507-aa72-d20aa8eb847b" (UID: "8422e67a-dd38-4507-aa72-d20aa8eb847b"). InnerVolumeSpecName "kube-api-access-62chd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:19:16.674933 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.674845    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62chd\" (UniqueName: \"kubernetes.io/projected/8422e67a-dd38-4507-aa72-d20aa8eb847b-kube-api-access-62chd\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:16.674933 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.674878    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8422e67a-dd38-4507-aa72-d20aa8eb847b-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:16.781455 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.781423    2527 generic.go:356] "Generic (PLEG): container finished" podID="8422e67a-dd38-4507-aa72-d20aa8eb847b" containerID="4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d" exitCode=0
Jul 09 20:19:16.781702 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.781474    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"
Jul 09 20:19:16.781702 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.781508    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b" event={"ID":"8422e67a-dd38-4507-aa72-d20aa8eb847b","Type":"ContainerDied","Data":"4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d"}
Jul 09 20:19:16.781702 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.781551    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b" event={"ID":"8422e67a-dd38-4507-aa72-d20aa8eb847b","Type":"ContainerDied","Data":"c8e1e9504e37fd480ca7889f4c4a55a248233d9b984754401fcac13fd6a27366"}
Jul 09 20:19:16.781702 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.781571    2527 scope.go:117] "RemoveContainer" containerID="4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d"
Jul 09 20:19:16.792339 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.792315    2527 scope.go:117] "RemoveContainer" containerID="4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d"
Jul 09 20:19:16.792636 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:19:16.792613    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d\": container with ID starting with 4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d not found: ID does not exist" containerID="4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d"
Jul 09 20:19:16.792748 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.792667    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d"} err="failed to get container status \"4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d\": rpc error: code = NotFound desc = could not find container \"4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d\": container with ID starting with 4bd8affcbc3ddc542fd1b4e2a0d45bbdbc4696babb92a2c8646c9f607a8dbe3d not found: ID does not exist"
Jul 09 20:19:16.803376 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.803341    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"]
Jul 09 20:19:16.807337 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:16.807314    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35f9ad-5cddf5d8d4-g647b"]
Jul 09 20:19:17.079235 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:17.079197    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8422e67a-dd38-4507-aa72-d20aa8eb847b" path="/var/lib/kubelet/pods/8422e67a-dd38-4507-aa72-d20aa8eb847b/volumes"
Jul 09 20:19:19.590378 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.590333    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66" podUID="b0157033-c319-4cab-a8d3-c42c45869892" containerName="main" containerID="cri-o://6f464668962799e6bfe7da5356a084941c2dfb8bead0f3a803dc32d57871e4ed" gracePeriod=45
Jul 09 20:19:19.800403 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.800369    2527 generic.go:356] "Generic (PLEG): container finished" podID="b0157033-c319-4cab-a8d3-c42c45869892" containerID="6f464668962799e6bfe7da5356a084941c2dfb8bead0f3a803dc32d57871e4ed" exitCode=0
Jul 09 20:19:19.800553 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.800460    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66" event={"ID":"b0157033-c319-4cab-a8d3-c42c45869892","Type":"ContainerDied","Data":"6f464668962799e6bfe7da5356a084941c2dfb8bead0f3a803dc32d57871e4ed"}
Jul 09 20:19:19.849391 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.849330    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:19:19.906981 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.906950    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-model-cache\") pod \"b0157033-c319-4cab-a8d3-c42c45869892\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") "
Jul 09 20:19:19.906981 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.906991    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0157033-c319-4cab-a8d3-c42c45869892-tls-certs\") pod \"b0157033-c319-4cab-a8d3-c42c45869892\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") "
Jul 09 20:19:19.907242 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907154    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-tmp-dir\") pod \"b0157033-c319-4cab-a8d3-c42c45869892\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") "
Jul 09 20:19:19.907242 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907186    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-home\") pod \"b0157033-c319-4cab-a8d3-c42c45869892\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") "
Jul 09 20:19:19.907242 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907214    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-dshm\") pod \"b0157033-c319-4cab-a8d3-c42c45869892\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") "
Jul 09 20:19:19.907410 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907236    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0157033-c319-4cab-a8d3-c42c45869892" (UID: "b0157033-c319-4cab-a8d3-c42c45869892"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:19.907410 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907249    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr9tj\" (UniqueName: \"kubernetes.io/projected/b0157033-c319-4cab-a8d3-c42c45869892-kube-api-access-fr9tj\") pod \"b0157033-c319-4cab-a8d3-c42c45869892\" (UID: \"b0157033-c319-4cab-a8d3-c42c45869892\") "
Jul 09 20:19:19.907498 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907415    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0157033-c319-4cab-a8d3-c42c45869892" (UID: "b0157033-c319-4cab-a8d3-c42c45869892"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:19.907498 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907450    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-home" (OuterVolumeSpecName: "home") pod "b0157033-c319-4cab-a8d3-c42c45869892" (UID: "b0157033-c319-4cab-a8d3-c42c45869892"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:19.907631 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907614    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:19.907693 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907639    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:19.907693 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.907671    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:19.909537 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.909513    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-dshm" (OuterVolumeSpecName: "dshm") pod "b0157033-c319-4cab-a8d3-c42c45869892" (UID: "b0157033-c319-4cab-a8d3-c42c45869892"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:19.909678 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.909550    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0157033-c319-4cab-a8d3-c42c45869892-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0157033-c319-4cab-a8d3-c42c45869892" (UID: "b0157033-c319-4cab-a8d3-c42c45869892"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:19:19.909678 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:19.909630    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0157033-c319-4cab-a8d3-c42c45869892-kube-api-access-fr9tj" (OuterVolumeSpecName: "kube-api-access-fr9tj") pod "b0157033-c319-4cab-a8d3-c42c45869892" (UID: "b0157033-c319-4cab-a8d3-c42c45869892"). InnerVolumeSpecName "kube-api-access-fr9tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:19:20.008411 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.008367    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0157033-c319-4cab-a8d3-c42c45869892-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:20.008411 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.008407    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr9tj\" (UniqueName: \"kubernetes.io/projected/b0157033-c319-4cab-a8d3-c42c45869892-kube-api-access-fr9tj\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:20.008617 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.008423    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0157033-c319-4cab-a8d3-c42c45869892-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:20.515414 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.515372    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk" podUID="cca4abc6-6662-44ad-9850-9c492e955ee7" containerName="main" containerID="cri-o://6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc" gracePeriod=45
Jul 09 20:19:20.775885 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.775826    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:19:20.811058 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.811019    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66" event={"ID":"b0157033-c319-4cab-a8d3-c42c45869892","Type":"ContainerDied","Data":"c488edc499e6f75f576b24a216ea73eea4ea9c3496f350fc443237402b701385"}
Jul 09 20:19:20.811244 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.811076    2527 scope.go:117] "RemoveContainer" containerID="6f464668962799e6bfe7da5356a084941c2dfb8bead0f3a803dc32d57871e4ed"
Jul 09 20:19:20.811244 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.811043    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"
Jul 09 20:19:20.812691 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.812666    2527 generic.go:356] "Generic (PLEG): container finished" podID="cca4abc6-6662-44ad-9850-9c492e955ee7" containerID="6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc" exitCode=0
Jul 09 20:19:20.812691 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.812729    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk" event={"ID":"cca4abc6-6662-44ad-9850-9c492e955ee7","Type":"ContainerDied","Data":"6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc"}
Jul 09 20:19:20.812691 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.812772    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk" event={"ID":"cca4abc6-6662-44ad-9850-9c492e955ee7","Type":"ContainerDied","Data":"932693fa5892b7abf511893458961aa14ead6c2531bb25171aa35ce1da0d5a0a"}
Jul 09 20:19:20.812691 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.812740    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"
Jul 09 20:19:20.822144 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.822123    2527 scope.go:117] "RemoveContainer" containerID="6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc"
Jul 09 20:19:20.831061 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.831039    2527 scope.go:117] "RemoveContainer" containerID="6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc"
Jul 09 20:19:20.831346 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:19:20.831325    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc\": container with ID starting with 6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc not found: ID does not exist" containerID="6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc"
Jul 09 20:19:20.831414 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.831353    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc"} err="failed to get container status \"6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc\": rpc error: code = NotFound desc = could not find container \"6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc\": container with ID starting with 6f956e01756854b2e07b4e51e709641245ffb945d5939107b8290964201f16bc not found: ID does not exist"
Jul 09 20:19:20.835960 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.835936    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"]
Jul 09 20:19:20.837840 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.837821    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-35f9ad/test-model-35f9ad-kserve-6dccbb9c97-dcj66"]
Jul 09 20:19:20.926700 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.926639    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mbxf\" (UniqueName: \"kubernetes.io/projected/cca4abc6-6662-44ad-9850-9c492e955ee7-kube-api-access-8mbxf\") pod \"cca4abc6-6662-44ad-9850-9c492e955ee7\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") "
Jul 09 20:19:20.927005 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.926985    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cca4abc6-6662-44ad-9850-9c492e955ee7-tls-certs\") pod \"cca4abc6-6662-44ad-9850-9c492e955ee7\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") "
Jul 09 20:19:20.927151 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.927138    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-tmp-dir\") pod \"cca4abc6-6662-44ad-9850-9c492e955ee7\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") "
Jul 09 20:19:20.927285 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.927273    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-dshm\") pod \"cca4abc6-6662-44ad-9850-9c492e955ee7\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") "
Jul 09 20:19:20.927414 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.927401    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-model-cache\") pod \"cca4abc6-6662-44ad-9850-9c492e955ee7\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") "
Jul 09 20:19:20.927518 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.927506    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-home\") pod \"cca4abc6-6662-44ad-9850-9c492e955ee7\" (UID: \"cca4abc6-6662-44ad-9850-9c492e955ee7\") "
Jul 09 20:19:20.927857 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.927825    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cca4abc6-6662-44ad-9850-9c492e955ee7" (UID: "cca4abc6-6662-44ad-9850-9c492e955ee7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:20.927966 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.927836    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-model-cache" (OuterVolumeSpecName: "model-cache") pod "cca4abc6-6662-44ad-9850-9c492e955ee7" (UID: "cca4abc6-6662-44ad-9850-9c492e955ee7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:20.928585 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.928172    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-home" (OuterVolumeSpecName: "home") pod "cca4abc6-6662-44ad-9850-9c492e955ee7" (UID: "cca4abc6-6662-44ad-9850-9c492e955ee7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:20.928703 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.928663    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:20.928703 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.928685    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:20.930244 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.930216    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cca4abc6-6662-44ad-9850-9c492e955ee7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cca4abc6-6662-44ad-9850-9c492e955ee7" (UID: "cca4abc6-6662-44ad-9850-9c492e955ee7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:19:20.930327 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.930252    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-dshm" (OuterVolumeSpecName: "dshm") pod "cca4abc6-6662-44ad-9850-9c492e955ee7" (UID: "cca4abc6-6662-44ad-9850-9c492e955ee7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:19:20.930327 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:20.930269    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca4abc6-6662-44ad-9850-9c492e955ee7-kube-api-access-8mbxf" (OuterVolumeSpecName: "kube-api-access-8mbxf") pod "cca4abc6-6662-44ad-9850-9c492e955ee7" (UID: "cca4abc6-6662-44ad-9850-9c492e955ee7"). InnerVolumeSpecName "kube-api-access-8mbxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:19:21.029443 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:21.029345    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cca4abc6-6662-44ad-9850-9c492e955ee7-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:21.029443 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:21.029370    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:21.029443 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:21.029379    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cca4abc6-6662-44ad-9850-9c492e955ee7-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:21.029443 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:21.029388    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mbxf\" (UniqueName: \"kubernetes.io/projected/cca4abc6-6662-44ad-9850-9c492e955ee7-kube-api-access-8mbxf\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:21.079583 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:21.079543    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0157033-c319-4cab-a8d3-c42c45869892" path="/var/lib/kubelet/pods/b0157033-c319-4cab-a8d3-c42c45869892/volumes"
Jul 09 20:19:21.129973 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:21.129940    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"]
Jul 09 20:19:21.133295 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:21.133267    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-54be63/test-model-54be63-kserve-65f9484dc5-cmcgk"]
Jul 09 20:19:23.078451 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:23.078419    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cca4abc6-6662-44ad-9850-9c492e955ee7" path="/var/lib/kubelet/pods/cca4abc6-6662-44ad-9850-9c492e955ee7/volumes"
Jul 09 20:19:31.903359 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903321    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"]
Jul 09 20:19:31.903816 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903799    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8422e67a-dd38-4507-aa72-d20aa8eb847b" containerName="maas-api"
Jul 09 20:19:31.903816 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903819    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="8422e67a-dd38-4507-aa72-d20aa8eb847b" containerName="maas-api"
Jul 09 20:19:31.903917 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903828    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0157033-c319-4cab-a8d3-c42c45869892" containerName="main"
Jul 09 20:19:31.903917 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903833    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0157033-c319-4cab-a8d3-c42c45869892" containerName="main"
Jul 09 20:19:31.903917 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903854    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cca4abc6-6662-44ad-9850-9c492e955ee7" containerName="main"
Jul 09 20:19:31.903917 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903862    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="cca4abc6-6662-44ad-9850-9c492e955ee7" containerName="main"
Jul 09 20:19:31.904041 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903935    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="cca4abc6-6662-44ad-9850-9c492e955ee7" containerName="main"
Jul 09 20:19:31.904041 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903946    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="8422e67a-dd38-4507-aa72-d20aa8eb847b" containerName="maas-api"
Jul 09 20:19:31.904041 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.903953    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0157033-c319-4cab-a8d3-c42c45869892" containerName="main"
Jul 09 20:19:31.906989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.906973    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"
Jul 09 20:19:31.909913 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.909895    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:19:31.916321 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:31.916293    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"]
Jul 09 20:19:32.050850 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.050813    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66b2b952-8a16-4164-afde-296e3e712507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj\" (UID: \"66b2b952-8a16-4164-afde-296e3e712507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"
Jul 09 20:19:32.151581 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.151542    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66b2b952-8a16-4164-afde-296e3e712507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj\" (UID: \"66b2b952-8a16-4164-afde-296e3e712507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"
Jul 09 20:19:32.152190 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.152171    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66b2b952-8a16-4164-afde-296e3e712507-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj\" (UID: \"66b2b952-8a16-4164-afde-296e3e712507\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"
Jul 09 20:19:32.219911 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.219814    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"
Jul 09 20:19:32.362164 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.362138    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"]
Jul 09 20:19:32.363872 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:19:32.363846    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b2b952_8a16_4164_afde_296e3e712507.slice/crio-bb4c7a299946d071a1258c52900316a1bb6f077825c247b9fe1bb961e82959f3 WatchSource:0}: Error finding container bb4c7a299946d071a1258c52900316a1bb6f077825c247b9fe1bb961e82959f3: Status 404 returned error can't find the container with id bb4c7a299946d071a1258c52900316a1bb6f077825c247b9fe1bb961e82959f3
Jul 09 20:19:32.880019 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.879922    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj" event={"ID":"66b2b952-8a16-4164-afde-296e3e712507","Type":"ContainerStarted","Data":"8f180afc153775e55145c531671fa8c77f0f45f86a516d91ac0393004e520a53"}
Jul 09 20:19:32.880019 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.879982    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj" event={"ID":"66b2b952-8a16-4164-afde-296e3e712507","Type":"ContainerStarted","Data":"bb4c7a299946d071a1258c52900316a1bb6f077825c247b9fe1bb961e82959f3"}
Jul 09 20:19:32.898225 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:32.898169    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj" podStartSLOduration=1.89815333 podStartE2EDuration="1.89815333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:19:32.89590582 +0000 UTC m=+2384.297040631" watchObservedRunningTime="2026-07-09 20:19:32.89815333 +0000 UTC m=+2384.299288141"
Jul 09 20:19:33.887452 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:33.887414    2527 generic.go:356] "Generic (PLEG): container finished" podID="66b2b952-8a16-4164-afde-296e3e712507" containerID="8f180afc153775e55145c531671fa8c77f0f45f86a516d91ac0393004e520a53" exitCode=0
Jul 09 20:19:33.887895 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:33.887477    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj" event={"ID":"66b2b952-8a16-4164-afde-296e3e712507","Type":"ContainerDied","Data":"8f180afc153775e55145c531671fa8c77f0f45f86a516d91ac0393004e520a53"}
Jul 09 20:19:35.025481 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:35.025456    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"
Jul 09 20:19:35.090002 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:35.089967    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66b2b952-8a16-4164-afde-296e3e712507-maas-api-service-ca\") pod \"66b2b952-8a16-4164-afde-296e3e712507\" (UID: \"66b2b952-8a16-4164-afde-296e3e712507\") "
Jul 09 20:19:35.090336 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:35.090315    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66b2b952-8a16-4164-afde-296e3e712507-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66b2b952-8a16-4164-afde-296e3e712507" (UID: "66b2b952-8a16-4164-afde-296e3e712507"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:19:35.191633 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:35.191550    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66b2b952-8a16-4164-afde-296e3e712507-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:35.899463 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:35.899434    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj"
Jul 09 20:19:35.899463 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:35.899454    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj" event={"ID":"66b2b952-8a16-4164-afde-296e3e712507","Type":"ContainerDied","Data":"bb4c7a299946d071a1258c52900316a1bb6f077825c247b9fe1bb961e82959f3"}
Jul 09 20:19:35.899689 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:35.899497    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb4c7a299946d071a1258c52900316a1bb6f077825c247b9fe1bb961e82959f3"
Jul 09 20:19:42.303264 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:42.303211    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"]
Jul 09 20:19:42.303749 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:42.303493    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2" podUID="4d2faf14-afd8-4253-8cf4-f5ec6b598df3" containerName="maas-api" containerID="cri-o://3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae" gracePeriod=30
Jul 09 20:19:43.264583 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.264537    2527 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2" podUID="4d2faf14-afd8-4253-8cf4-f5ec6b598df3" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.47:8443/health\": dial tcp 10.134.0.47:8443: connect: connection refused"
Jul 09 20:19:43.553078 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.553045    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:19:43.672985 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.672944    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-maas-api-tls\") pod \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") "
Jul 09 20:19:43.673187 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.673068    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-openshift-service-ca\") pod \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") "
Jul 09 20:19:43.673187 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.673093    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpvpq\" (UniqueName: \"kubernetes.io/projected/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-kube-api-access-rpvpq\") pod \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\" (UID: \"4d2faf14-afd8-4253-8cf4-f5ec6b598df3\") "
Jul 09 20:19:43.673503 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.673475    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d2faf14-afd8-4253-8cf4-f5ec6b598df3" (UID: "4d2faf14-afd8-4253-8cf4-f5ec6b598df3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:19:43.675286 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.675251    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d2faf14-afd8-4253-8cf4-f5ec6b598df3" (UID: "4d2faf14-afd8-4253-8cf4-f5ec6b598df3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:19:43.675286 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.675254    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-kube-api-access-rpvpq" (OuterVolumeSpecName: "kube-api-access-rpvpq") pod "4d2faf14-afd8-4253-8cf4-f5ec6b598df3" (UID: "4d2faf14-afd8-4253-8cf4-f5ec6b598df3"). InnerVolumeSpecName "kube-api-access-rpvpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:19:43.773967 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.773914    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:43.773967 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.773964    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:43.773967 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.773974    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpvpq\" (UniqueName: \"kubernetes.io/projected/4d2faf14-afd8-4253-8cf4-f5ec6b598df3-kube-api-access-rpvpq\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:19:43.942145 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.942051    2527 generic.go:356] "Generic (PLEG): container finished" podID="4d2faf14-afd8-4253-8cf4-f5ec6b598df3" containerID="3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae" exitCode=0
Jul 09 20:19:43.942145 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.942097    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"
Jul 09 20:19:43.942145 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.942134    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2" event={"ID":"4d2faf14-afd8-4253-8cf4-f5ec6b598df3","Type":"ContainerDied","Data":"3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae"}
Jul 09 20:19:43.942408 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.942188    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2" event={"ID":"4d2faf14-afd8-4253-8cf4-f5ec6b598df3","Type":"ContainerDied","Data":"9c58ab2242abbd7ae4e1e70ce42ed9a80e2a992654a7f37a8e0501ecb84ed815"}
Jul 09 20:19:43.942408 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.942209    2527 scope.go:117] "RemoveContainer" containerID="3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae"
Jul 09 20:19:43.952635 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.952619    2527 scope.go:117] "RemoveContainer" containerID="3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae"
Jul 09 20:19:43.952953 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:19:43.952932    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae\": container with ID starting with 3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae not found: ID does not exist" containerID="3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae"
Jul 09 20:19:43.952998 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.952963    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae"} err="failed to get container status \"3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae\": rpc error: code = NotFound desc = could not find container \"3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae\": container with ID starting with 3093e129d6c69b378d83f1e133c0a814b443b23b38a49731e54260937e4517ae not found: ID does not exist"
Jul 09 20:19:43.964342 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.964317    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"]
Jul 09 20:19:43.970392 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:43.970368    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-54be63-7bcccfc444-6crj2"]
Jul 09 20:19:45.077781 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:19:45.077741    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d2faf14-afd8-4253-8cf4-f5ec6b598df3" path="/var/lib/kubelet/pods/4d2faf14-afd8-4253-8cf4-f5ec6b598df3/volumes"
Jul 09 20:20:01.701931 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.701892    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"]
Jul 09 20:20:01.702490 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.702474    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d2faf14-afd8-4253-8cf4-f5ec6b598df3" containerName="maas-api"
Jul 09 20:20:01.702542 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.702494    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d2faf14-afd8-4253-8cf4-f5ec6b598df3" containerName="maas-api"
Jul 09 20:20:01.702542 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.702512    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66b2b952-8a16-4164-afde-296e3e712507" containerName="revoke-keys"
Jul 09 20:20:01.702542 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.702521    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="66b2b952-8a16-4164-afde-296e3e712507" containerName="revoke-keys"
Jul 09 20:20:01.702669 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.702639    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d2faf14-afd8-4253-8cf4-f5ec6b598df3" containerName="maas-api"
Jul 09 20:20:01.702714 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.702675    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="66b2b952-8a16-4164-afde-296e3e712507" containerName="revoke-keys"
Jul 09 20:20:01.706178 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.706158    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.708870 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.708845    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:20:01.708870 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.708867    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:20:01.709893 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.709872    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b7d55f-serving-cert\""
Jul 09 20:20:01.710001 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.709895    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:20:01.713783 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.713761    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"]
Jul 09 20:20:01.857299 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.857266    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.857492 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.857322    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4fz\" (UniqueName: \"kubernetes.io/projected/d6f92438-ca1d-4010-b703-75bd7cd044a2-kube-api-access-5t4fz\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.857492 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.857367    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f92438-ca1d-4010-b703-75bd7cd044a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.958998 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.958894    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.958998 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:20:01.958977    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-serving-cert: secret "maas-api-e2e-api-a-b7d55f-serving-cert" not found
Jul 09 20:20:01.959233 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.958985    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4fz\" (UniqueName: \"kubernetes.io/projected/d6f92438-ca1d-4010-b703-75bd7cd044a2-kube-api-access-5t4fz\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.959233 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:20:01.959061    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls podName:d6f92438-ca1d-4010-b703-75bd7cd044a2 nodeName:}" failed. No retries permitted until 2026-07-09 20:20:02.459038927 +0000 UTC m=+2413.860173737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls") pod "maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9" (UID: "d6f92438-ca1d-4010-b703-75bd7cd044a2") : secret "maas-api-e2e-api-a-b7d55f-serving-cert" not found
Jul 09 20:20:01.959233 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.959119    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f92438-ca1d-4010-b703-75bd7cd044a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.959735 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.959716    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f92438-ca1d-4010-b703-75bd7cd044a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:01.968202 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:01.968183    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4fz\" (UniqueName: \"kubernetes.io/projected/d6f92438-ca1d-4010-b703-75bd7cd044a2-kube-api-access-5t4fz\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:02.464479 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:02.464431    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:02.466995 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:02.466964    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls\") pod \"maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:02.618720 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:02.618665    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:02.784455 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:02.784420    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"]
Jul 09 20:20:02.786361 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:20:02.786335    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f92438_ca1d_4010_b703_75bd7cd044a2.slice/crio-399a9879e33eb61f00f792f6ab1255791e0d15ca01884002397c585730009884 WatchSource:0}: Error finding container 399a9879e33eb61f00f792f6ab1255791e0d15ca01884002397c585730009884: Status 404 returned error can't find the container with id 399a9879e33eb61f00f792f6ab1255791e0d15ca01884002397c585730009884
Jul 09 20:20:03.040419 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:03.040330    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9" event={"ID":"d6f92438-ca1d-4010-b703-75bd7cd044a2","Type":"ContainerStarted","Data":"399a9879e33eb61f00f792f6ab1255791e0d15ca01884002397c585730009884"}
Jul 09 20:20:04.047012 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:04.046971    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9" event={"ID":"d6f92438-ca1d-4010-b703-75bd7cd044a2","Type":"ContainerStarted","Data":"471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b"}
Jul 09 20:20:04.047403 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:04.047074    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:04.065767 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:04.065705    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9" podStartSLOduration=2.349302294 podStartE2EDuration="3.065675495s" totalImagesPullingTime="716.373201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:20:04.064560854 +0000 UTC m=+2415.465695664" watchObservedRunningTime="2026-07-09 20:20:04.065675495 +0000 UTC m=+2415.466810307"
Jul 09 20:20:09.790146 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.790056    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"]
Jul 09 20:20:09.794637 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.794615    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.797299 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.797275    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e062b5-serving-cert\""
Jul 09 20:20:09.806157 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.806131    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"]
Jul 09 20:20:09.837459 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.837423    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8176d503-6674-44c5-ac5b-98743b17ed7e-openshift-service-ca\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.837633 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.837527    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g876c\" (UniqueName: \"kubernetes.io/projected/8176d503-6674-44c5-ac5b-98743b17ed7e-kube-api-access-g876c\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.837633 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.837599    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.938882 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.938841    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8176d503-6674-44c5-ac5b-98743b17ed7e-openshift-service-ca\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.939107 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.938916    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g876c\" (UniqueName: \"kubernetes.io/projected/8176d503-6674-44c5-ac5b-98743b17ed7e-kube-api-access-g876c\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.939107 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.938954    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.939107 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:20:09.939069    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-serving-cert: secret "maas-api-e2e-api-b-e062b5-serving-cert" not found
Jul 09 20:20:09.939297 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:20:09.939129    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls podName:8176d503-6674-44c5-ac5b-98743b17ed7e nodeName:}" failed. No retries permitted until 2026-07-09 20:20:10.439108293 +0000 UTC m=+2421.840243081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls") pod "maas-api-e2e-api-b-e062b5-789ccc4566-dk84d" (UID: "8176d503-6674-44c5-ac5b-98743b17ed7e") : secret "maas-api-e2e-api-b-e062b5-serving-cert" not found
Jul 09 20:20:09.939537 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.939516    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8176d503-6674-44c5-ac5b-98743b17ed7e-openshift-service-ca\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:09.949834 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:09.949810    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g876c\" (UniqueName: \"kubernetes.io/projected/8176d503-6674-44c5-ac5b-98743b17ed7e-kube-api-access-g876c\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:10.058723 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:10.058620    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:10.444584 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:10.444474    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:10.447276 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:10.447241    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls\") pod \"maas-api-e2e-api-b-e062b5-789ccc4566-dk84d\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:10.706335 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:10.706239    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:10.848264 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:10.848236    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"]
Jul 09 20:20:10.849929 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:20:10.849907    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8176d503_6674_44c5_ac5b_98743b17ed7e.slice/crio-a273ae50558c1cec8c4faaee28816edb98b26ff959cf31d86c76d4d4f56034bb WatchSource:0}: Error finding container a273ae50558c1cec8c4faaee28816edb98b26ff959cf31d86c76d4d4f56034bb: Status 404 returned error can't find the container with id a273ae50558c1cec8c4faaee28816edb98b26ff959cf31d86c76d4d4f56034bb
Jul 09 20:20:11.085348 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:11.085315    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d" event={"ID":"8176d503-6674-44c5-ac5b-98743b17ed7e","Type":"ContainerStarted","Data":"a273ae50558c1cec8c4faaee28816edb98b26ff959cf31d86c76d4d4f56034bb"}
Jul 09 20:20:12.092689 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:12.092626    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d" event={"ID":"8176d503-6674-44c5-ac5b-98743b17ed7e","Type":"ContainerStarted","Data":"92db7452a5a7e77a2430d35bfaa64d47735bf99c3b46387c957cab1d379d3c8e"}
Jul 09 20:20:12.093129 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:12.092717    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:12.109868 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:12.109814    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d" podStartSLOduration=2.667681724 podStartE2EDuration="3.109795909s" totalImagesPullingTime="442.114185ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:20:12.108114976 +0000 UTC m=+2423.509249787" watchObservedRunningTime="2026-07-09 20:20:12.109795909 +0000 UTC m=+2423.510930722"
Jul 09 20:20:18.103783 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:18.103754    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:22.309560 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.309521    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"]
Jul 09 20:20:22.313160 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.313140    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"
Jul 09 20:20:22.315908 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.315885    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:20:22.319747 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.319720    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"]
Jul 09 20:20:22.365576 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.365538    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9247b046-27f5-4c11-b920-bc70a626c621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2\" (UID: \"9247b046-27f5-4c11-b920-bc70a626c621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"
Jul 09 20:20:22.466547 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.466509    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9247b046-27f5-4c11-b920-bc70a626c621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2\" (UID: \"9247b046-27f5-4c11-b920-bc70a626c621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"
Jul 09 20:20:22.467123 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.467105    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9247b046-27f5-4c11-b920-bc70a626c621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2\" (UID: \"9247b046-27f5-4c11-b920-bc70a626c621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"
Jul 09 20:20:22.626835 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.626742    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"
Jul 09 20:20:22.769023 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:22.768998    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"]
Jul 09 20:20:22.771362 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:20:22.771331    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9247b046_27f5_4c11_b920_bc70a626c621.slice/crio-7df146c14628ed532552c46d607e8bac4d979ac52ad81486f69811c57eac4170 WatchSource:0}: Error finding container 7df146c14628ed532552c46d607e8bac4d979ac52ad81486f69811c57eac4170: Status 404 returned error can't find the container with id 7df146c14628ed532552c46d607e8bac4d979ac52ad81486f69811c57eac4170
Jul 09 20:20:23.150273 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:23.150239    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2" event={"ID":"9247b046-27f5-4c11-b920-bc70a626c621","Type":"ContainerStarted","Data":"7df146c14628ed532552c46d607e8bac4d979ac52ad81486f69811c57eac4170"}
Jul 09 20:20:24.156431 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:24.156396    2527 generic.go:356] "Generic (PLEG): container finished" podID="9247b046-27f5-4c11-b920-bc70a626c621" containerID="2953c475fd143f39a807e5fdf9f1c454aa7ca2eeea76928a679f3c170cc4d71f" exitCode=0
Jul 09 20:20:24.156833 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:24.156453    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2" event={"ID":"9247b046-27f5-4c11-b920-bc70a626c621","Type":"ContainerDied","Data":"2953c475fd143f39a807e5fdf9f1c454aa7ca2eeea76928a679f3c170cc4d71f"}
Jul 09 20:20:25.298535 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:25.298513    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"
Jul 09 20:20:25.394718 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:25.394683    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9247b046-27f5-4c11-b920-bc70a626c621-maas-api-service-ca\") pod \"9247b046-27f5-4c11-b920-bc70a626c621\" (UID: \"9247b046-27f5-4c11-b920-bc70a626c621\") "
Jul 09 20:20:25.395053 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:25.395027    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9247b046-27f5-4c11-b920-bc70a626c621-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9247b046-27f5-4c11-b920-bc70a626c621" (UID: "9247b046-27f5-4c11-b920-bc70a626c621"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:20:25.496247 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:25.496118    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9247b046-27f5-4c11-b920-bc70a626c621-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:20:26.167194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:26.167159    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2"
Jul 09 20:20:26.167366 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:26.167165    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2" event={"ID":"9247b046-27f5-4c11-b920-bc70a626c621","Type":"ContainerDied","Data":"7df146c14628ed532552c46d607e8bac4d979ac52ad81486f69811c57eac4170"}
Jul 09 20:20:26.167366 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:26.167267    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7df146c14628ed532552c46d607e8bac4d979ac52ad81486f69811c57eac4170"
Jul 09 20:20:32.790762 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:32.790699    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"]
Jul 09 20:20:32.791252 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:32.791001    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9" podUID="d6f92438-ca1d-4010-b703-75bd7cd044a2" containerName="maas-api" containerID="cri-o://471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b" gracePeriod=30
Jul 09 20:20:33.054036 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.053965    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:33.171448 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.171409    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls\") pod \"d6f92438-ca1d-4010-b703-75bd7cd044a2\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") "
Jul 09 20:20:33.171676 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.171505    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f92438-ca1d-4010-b703-75bd7cd044a2-openshift-service-ca\") pod \"d6f92438-ca1d-4010-b703-75bd7cd044a2\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") "
Jul 09 20:20:33.171676 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.171579    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t4fz\" (UniqueName: \"kubernetes.io/projected/d6f92438-ca1d-4010-b703-75bd7cd044a2-kube-api-access-5t4fz\") pod \"d6f92438-ca1d-4010-b703-75bd7cd044a2\" (UID: \"d6f92438-ca1d-4010-b703-75bd7cd044a2\") "
Jul 09 20:20:33.171968 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.171936    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6f92438-ca1d-4010-b703-75bd7cd044a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6f92438-ca1d-4010-b703-75bd7cd044a2" (UID: "d6f92438-ca1d-4010-b703-75bd7cd044a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:20:33.173876 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.173850    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6f92438-ca1d-4010-b703-75bd7cd044a2-kube-api-access-5t4fz" (OuterVolumeSpecName: "kube-api-access-5t4fz") pod "d6f92438-ca1d-4010-b703-75bd7cd044a2" (UID: "d6f92438-ca1d-4010-b703-75bd7cd044a2"). InnerVolumeSpecName "kube-api-access-5t4fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:20:33.173965 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.173900    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6f92438-ca1d-4010-b703-75bd7cd044a2" (UID: "d6f92438-ca1d-4010-b703-75bd7cd044a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:20:33.205706 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.205630    2527 generic.go:356] "Generic (PLEG): container finished" podID="d6f92438-ca1d-4010-b703-75bd7cd044a2" containerID="471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b" exitCode=0
Jul 09 20:20:33.205706 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.205703    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"
Jul 09 20:20:33.205919 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.205719    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9" event={"ID":"d6f92438-ca1d-4010-b703-75bd7cd044a2","Type":"ContainerDied","Data":"471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b"}
Jul 09 20:20:33.205919 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.205777    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9" event={"ID":"d6f92438-ca1d-4010-b703-75bd7cd044a2","Type":"ContainerDied","Data":"399a9879e33eb61f00f792f6ab1255791e0d15ca01884002397c585730009884"}
Jul 09 20:20:33.205919 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.205802    2527 scope.go:117] "RemoveContainer" containerID="471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b"
Jul 09 20:20:33.215257 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.215238    2527 scope.go:117] "RemoveContainer" containerID="471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b"
Jul 09 20:20:33.215553 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:20:33.215534    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b\": container with ID starting with 471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b not found: ID does not exist" containerID="471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b"
Jul 09 20:20:33.215605 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.215563    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b"} err="failed to get container status \"471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b\": rpc error: code = NotFound desc = could not find container \"471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b\": container with ID starting with 471c2960e2f40a87368156b15b701ea91bd0eb712d9dcaccafa97d58dd29f93b not found: ID does not exist"
Jul 09 20:20:33.228343 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.228296    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"]
Jul 09 20:20:33.230551 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.230526    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b7d55f-6bcb54d66b-5l6w9"]
Jul 09 20:20:33.272494 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.272453    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t4fz\" (UniqueName: \"kubernetes.io/projected/d6f92438-ca1d-4010-b703-75bd7cd044a2-kube-api-access-5t4fz\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:20:33.272494 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.272486    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6f92438-ca1d-4010-b703-75bd7cd044a2-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:20:33.272494 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:33.272501    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f92438-ca1d-4010-b703-75bd7cd044a2-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:20:35.077416 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:35.077382    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6f92438-ca1d-4010-b703-75bd7cd044a2" path="/var/lib/kubelet/pods/d6f92438-ca1d-4010-b703-75bd7cd044a2/volumes"
Jul 09 20:20:48.442271 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.442236    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"]
Jul 09 20:20:48.442734 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.442656    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9247b046-27f5-4c11-b920-bc70a626c621" containerName="revoke-keys"
Jul 09 20:20:48.442734 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.442671    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="9247b046-27f5-4c11-b920-bc70a626c621" containerName="revoke-keys"
Jul 09 20:20:48.442734 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.442693    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6f92438-ca1d-4010-b703-75bd7cd044a2" containerName="maas-api"
Jul 09 20:20:48.442734 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.442700    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6f92438-ca1d-4010-b703-75bd7cd044a2" containerName="maas-api"
Jul 09 20:20:48.442867 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.442769    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6f92438-ca1d-4010-b703-75bd7cd044a2" containerName="maas-api"
Jul 09 20:20:48.442867 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.442780    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="9247b046-27f5-4c11-b920-bc70a626c621" containerName="revoke-keys"
Jul 09 20:20:48.447194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.447172    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"
Jul 09 20:20:48.449904 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.449882    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:20:48.454259 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.454235    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"]
Jul 09 20:20:48.522136 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.522093    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb1ca59-cce8-4c35-9882-70e720376908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm\" (UID: \"feb1ca59-cce8-4c35-9882-70e720376908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"
Jul 09 20:20:48.622892 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.622859    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb1ca59-cce8-4c35-9882-70e720376908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm\" (UID: \"feb1ca59-cce8-4c35-9882-70e720376908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"
Jul 09 20:20:48.623494 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.623475    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb1ca59-cce8-4c35-9882-70e720376908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm\" (UID: \"feb1ca59-cce8-4c35-9882-70e720376908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"
Jul 09 20:20:48.759543 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.759426    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"
Jul 09 20:20:48.901410 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:48.901382    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"]
Jul 09 20:20:48.904126 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:20:48.904104    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeb1ca59_cce8_4c35_9882_70e720376908.slice/crio-3ff61f874684669638e6b019071ee44965ea038bb45f9fb1a4848f09e1f5438c WatchSource:0}: Error finding container 3ff61f874684669638e6b019071ee44965ea038bb45f9fb1a4848f09e1f5438c: Status 404 returned error can't find the container with id 3ff61f874684669638e6b019071ee44965ea038bb45f9fb1a4848f09e1f5438c
Jul 09 20:20:49.293109 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:49.293067    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm" event={"ID":"feb1ca59-cce8-4c35-9882-70e720376908","Type":"ContainerStarted","Data":"718aa6dc74e7d4d39982024b2fb5072e92cc421ef714c2323c57ea5b32c2454a"}
Jul 09 20:20:49.293288 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:49.293120    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm" event={"ID":"feb1ca59-cce8-4c35-9882-70e720376908","Type":"ContainerStarted","Data":"3ff61f874684669638e6b019071ee44965ea038bb45f9fb1a4848f09e1f5438c"}
Jul 09 20:20:49.312512 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:49.312462    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm" podStartSLOduration=1.312446531 podStartE2EDuration="1.312446531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:20:49.310349523 +0000 UTC m=+2460.711484335" watchObservedRunningTime="2026-07-09 20:20:49.312446531 +0000 UTC m=+2460.713581342"
Jul 09 20:20:50.301016 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:50.300984    2527 generic.go:356] "Generic (PLEG): container finished" podID="feb1ca59-cce8-4c35-9882-70e720376908" containerID="718aa6dc74e7d4d39982024b2fb5072e92cc421ef714c2323c57ea5b32c2454a" exitCode=0
Jul 09 20:20:50.301396 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:50.301077    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm" event={"ID":"feb1ca59-cce8-4c35-9882-70e720376908","Type":"ContainerDied","Data":"718aa6dc74e7d4d39982024b2fb5072e92cc421ef714c2323c57ea5b32c2454a"}
Jul 09 20:20:51.445009 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:51.444984    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"
Jul 09 20:20:51.550456 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:51.550419    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb1ca59-cce8-4c35-9882-70e720376908-maas-api-service-ca\") pod \"feb1ca59-cce8-4c35-9882-70e720376908\" (UID: \"feb1ca59-cce8-4c35-9882-70e720376908\") "
Jul 09 20:20:51.550819 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:51.550798    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feb1ca59-cce8-4c35-9882-70e720376908-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "feb1ca59-cce8-4c35-9882-70e720376908" (UID: "feb1ca59-cce8-4c35-9882-70e720376908"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:20:51.651523 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:51.651442    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb1ca59-cce8-4c35-9882-70e720376908-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:20:52.313127 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:52.313097    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm"
Jul 09 20:20:52.313336 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:52.313139    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm" event={"ID":"feb1ca59-cce8-4c35-9882-70e720376908","Type":"ContainerDied","Data":"3ff61f874684669638e6b019071ee44965ea038bb45f9fb1a4848f09e1f5438c"}
Jul 09 20:20:52.313336 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:52.313187    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ff61f874684669638e6b019071ee44965ea038bb45f9fb1a4848f09e1f5438c"
Jul 09 20:20:59.071333 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.071278    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"]
Jul 09 20:20:59.075670 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.075593    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d" podUID="8176d503-6674-44c5-ac5b-98743b17ed7e" containerName="maas-api" containerID="cri-o://92db7452a5a7e77a2430d35bfaa64d47735bf99c3b46387c957cab1d379d3c8e" gracePeriod=30
Jul 09 20:20:59.349941 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.349905    2527 generic.go:356] "Generic (PLEG): container finished" podID="8176d503-6674-44c5-ac5b-98743b17ed7e" containerID="92db7452a5a7e77a2430d35bfaa64d47735bf99c3b46387c957cab1d379d3c8e" exitCode=0
Jul 09 20:20:59.350077 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.349979    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d" event={"ID":"8176d503-6674-44c5-ac5b-98743b17ed7e","Type":"ContainerDied","Data":"92db7452a5a7e77a2430d35bfaa64d47735bf99c3b46387c957cab1d379d3c8e"}
Jul 09 20:20:59.350077 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.350042    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d" event={"ID":"8176d503-6674-44c5-ac5b-98743b17ed7e","Type":"ContainerDied","Data":"a273ae50558c1cec8c4faaee28816edb98b26ff959cf31d86c76d4d4f56034bb"}
Jul 09 20:20:59.350077 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.350066    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a273ae50558c1cec8c4faaee28816edb98b26ff959cf31d86c76d4d4f56034bb"
Jul 09 20:20:59.350189 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.350089    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:20:59.431113 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.431085    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8176d503-6674-44c5-ac5b-98743b17ed7e-openshift-service-ca\") pod \"8176d503-6674-44c5-ac5b-98743b17ed7e\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") "
Jul 09 20:20:59.431281 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.431153    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls\") pod \"8176d503-6674-44c5-ac5b-98743b17ed7e\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") "
Jul 09 20:20:59.431281 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.431179    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g876c\" (UniqueName: \"kubernetes.io/projected/8176d503-6674-44c5-ac5b-98743b17ed7e-kube-api-access-g876c\") pod \"8176d503-6674-44c5-ac5b-98743b17ed7e\" (UID: \"8176d503-6674-44c5-ac5b-98743b17ed7e\") "
Jul 09 20:20:59.431517 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.431490    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8176d503-6674-44c5-ac5b-98743b17ed7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8176d503-6674-44c5-ac5b-98743b17ed7e" (UID: "8176d503-6674-44c5-ac5b-98743b17ed7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:20:59.433451 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.433429    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8176d503-6674-44c5-ac5b-98743b17ed7e-kube-api-access-g876c" (OuterVolumeSpecName: "kube-api-access-g876c") pod "8176d503-6674-44c5-ac5b-98743b17ed7e" (UID: "8176d503-6674-44c5-ac5b-98743b17ed7e"). InnerVolumeSpecName "kube-api-access-g876c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:20:59.433572 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.433492    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8176d503-6674-44c5-ac5b-98743b17ed7e" (UID: "8176d503-6674-44c5-ac5b-98743b17ed7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:20:59.532330 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.532292    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8176d503-6674-44c5-ac5b-98743b17ed7e-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:20:59.532330 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.532327    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g876c\" (UniqueName: \"kubernetes.io/projected/8176d503-6674-44c5-ac5b-98743b17ed7e-kube-api-access-g876c\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:20:59.532330 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:20:59.532337    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8176d503-6674-44c5-ac5b-98743b17ed7e-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:21:00.354691 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:00.354661    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"
Jul 09 20:21:00.376294 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:00.376265    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"]
Jul 09 20:21:00.380986 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:00.380956    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e062b5-789ccc4566-dk84d"]
Jul 09 20:21:01.076856 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:01.076822    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8176d503-6674-44c5-ac5b-98743b17ed7e" path="/var/lib/kubelet/pods/8176d503-6674-44c5-ac5b-98743b17ed7e/volumes"
Jul 09 20:21:18.013834 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.013802    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"]
Jul 09 20:21:18.014441 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.014422    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8176d503-6674-44c5-ac5b-98743b17ed7e" containerName="maas-api"
Jul 09 20:21:18.014441 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.014442    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="8176d503-6674-44c5-ac5b-98743b17ed7e" containerName="maas-api"
Jul 09 20:21:18.014604 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.014456    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feb1ca59-cce8-4c35-9882-70e720376908" containerName="revoke-keys"
Jul 09 20:21:18.014604 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.014465    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="feb1ca59-cce8-4c35-9882-70e720376908" containerName="revoke-keys"
Jul 09 20:21:18.014604 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.014562    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="8176d503-6674-44c5-ac5b-98743b17ed7e" containerName="maas-api"
Jul 09 20:21:18.014604 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.014575    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="feb1ca59-cce8-4c35-9882-70e720376908" containerName="revoke-keys"
Jul 09 20:21:18.018255 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.018234    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.020964 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.020940    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 20:21:18.021059 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.020968    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-7b406b-serving-cert\""
Jul 09 20:21:18.021124 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.021100    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d9hhm\""
Jul 09 20:21:18.022154 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.022139    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 20:21:18.028526 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.028502    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"]
Jul 09 20:21:18.120232 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.120190    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e088a395-f669-4554-beff-af04fbbf639f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.120671 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.120613    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22t6k\" (UniqueName: \"kubernetes.io/projected/e088a395-f669-4554-beff-af04fbbf639f-kube-api-access-22t6k\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.120906 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.120882    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e088a395-f669-4554-beff-af04fbbf639f-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.221586 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.221548    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e088a395-f669-4554-beff-af04fbbf639f-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.221818 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.221657    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e088a395-f669-4554-beff-af04fbbf639f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.221818 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.221734    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22t6k\" (UniqueName: \"kubernetes.io/projected/e088a395-f669-4554-beff-af04fbbf639f-kube-api-access-22t6k\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.222282 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.222262    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e088a395-f669-4554-beff-af04fbbf639f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.224097 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.224076    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e088a395-f669-4554-beff-af04fbbf639f-maas-api-tls\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.230996 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.230970    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22t6k\" (UniqueName: \"kubernetes.io/projected/e088a395-f669-4554-beff-af04fbbf639f-kube-api-access-22t6k\") pod \"maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.329789 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.329752    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:18.473221 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:18.473190    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"]
Jul 09 20:21:18.474784 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:21:18.474756    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode088a395_f669_4554_beff_af04fbbf639f.slice/crio-471cc4579b501cc7097a8bbe4d6b76ad14bd1f24926cdd73a1226686295b5053 WatchSource:0}: Error finding container 471cc4579b501cc7097a8bbe4d6b76ad14bd1f24926cdd73a1226686295b5053: Status 404 returned error can't find the container with id 471cc4579b501cc7097a8bbe4d6b76ad14bd1f24926cdd73a1226686295b5053
Jul 09 20:21:19.453850 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:19.453811    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m" event={"ID":"e088a395-f669-4554-beff-af04fbbf639f","Type":"ContainerStarted","Data":"d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d"}
Jul 09 20:21:19.454289 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:19.453871    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m" event={"ID":"e088a395-f669-4554-beff-af04fbbf639f","Type":"ContainerStarted","Data":"471cc4579b501cc7097a8bbe4d6b76ad14bd1f24926cdd73a1226686295b5053"}
Jul 09 20:21:19.454289 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:19.453899    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:19.474691 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:19.474624    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m" podStartSLOduration=2.107616321 podStartE2EDuration="2.474606681s" totalImagesPullingTime="366.99036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:21:19.47221223 +0000 UTC m=+2490.873347044" watchObservedRunningTime="2026-07-09 20:21:19.474606681 +0000 UTC m=+2490.875741492"
Jul 09 20:21:25.465400 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:25.465370    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:21:26.140746 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.140715    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"]
Jul 09 20:21:26.145087 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.145063    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.147861 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.147830    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3452a7-serving-cert\""
Jul 09 20:21:26.156741 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.156719    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"]
Jul 09 20:21:26.203807 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.203766    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.203987 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.203828    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jjfd\" (UniqueName: \"kubernetes.io/projected/83de1a9d-2b43-4f15-a11d-d8eb9b894054-kube-api-access-5jjfd\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.203987 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.203936    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83de1a9d-2b43-4f15-a11d-d8eb9b894054-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.305366 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.305329    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83de1a9d-2b43-4f15-a11d-d8eb9b894054-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.305568 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.305393    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.305568 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.305434    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jjfd\" (UniqueName: \"kubernetes.io/projected/83de1a9d-2b43-4f15-a11d-d8eb9b894054-kube-api-access-5jjfd\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.305723 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:21:26.305562    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-serving-cert: secret "maas-api-e2e-shared-b-3452a7-serving-cert" not found
Jul 09 20:21:26.305723 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:21:26.305678    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls podName:83de1a9d-2b43-4f15-a11d-d8eb9b894054 nodeName:}" failed. No retries permitted until 2026-07-09 20:21:26.8056225 +0000 UTC m=+2498.206757295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls") pod "maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx" (UID: "83de1a9d-2b43-4f15-a11d-d8eb9b894054") : secret "maas-api-e2e-shared-b-3452a7-serving-cert" not found
Jul 09 20:21:26.306089 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.306068    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83de1a9d-2b43-4f15-a11d-d8eb9b894054-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.321178 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.321150    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jjfd\" (UniqueName: \"kubernetes.io/projected/83de1a9d-2b43-4f15-a11d-d8eb9b894054-kube-api-access-5jjfd\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.810422 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.810375    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:26.813170 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:26.813143    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls\") pod \"maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:27.057703 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:27.057639    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:27.196249 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:27.196221    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"]
Jul 09 20:21:27.197938 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:21:27.197906    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83de1a9d_2b43_4f15_a11d_d8eb9b894054.slice/crio-5a66ec85dc1c3b58d02f6db37962801066378a3d37ef93ef12c48f9797c63893 WatchSource:0}: Error finding container 5a66ec85dc1c3b58d02f6db37962801066378a3d37ef93ef12c48f9797c63893: Status 404 returned error can't find the container with id 5a66ec85dc1c3b58d02f6db37962801066378a3d37ef93ef12c48f9797c63893
Jul 09 20:21:27.501261 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:27.501170    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx" event={"ID":"83de1a9d-2b43-4f15-a11d-d8eb9b894054","Type":"ContainerStarted","Data":"5a66ec85dc1c3b58d02f6db37962801066378a3d37ef93ef12c48f9797c63893"}
Jul 09 20:21:28.508230 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:28.508183    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx" event={"ID":"83de1a9d-2b43-4f15-a11d-d8eb9b894054","Type":"ContainerStarted","Data":"37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0"}
Jul 09 20:21:28.508744 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:28.508268    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:28.525873 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:28.525817    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx" podStartSLOduration=2.185174243 podStartE2EDuration="2.525796323s" totalImagesPullingTime="340.62208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:21:28.524757636 +0000 UTC m=+2499.925892445" watchObservedRunningTime="2026-07-09 20:21:28.525796323 +0000 UTC m=+2499.926931135"
Jul 09 20:21:34.519723 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:34.519697    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:21:47.687944 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.687911    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"]
Jul 09 20:21:47.691748 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.691726    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.695698 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.695674    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"auth-test-model-7b406b-kserve-self-signed-certs\""
Jul 09 20:21:47.695848 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.695679    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"default-dockercfg-mlp92\""
Jul 09 20:21:47.695848 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.695705    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"openshift-service-ca.crt\""
Jul 09 20:21:47.695848 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.695720    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"kube-root-ca.crt\""
Jul 09 20:21:47.700594 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.700571    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"]
Jul 09 20:21:47.727922 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.727888    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424659c2-e71b-4390-9eb3-1c8dce8df887-tls-certs\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.728091 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.727964    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-tmp-dir\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.728091 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.727985    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-home\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.728091 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.728026    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-dshm\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.728091 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.728043    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-model-cache\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.728091 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.728066    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpmp9\" (UniqueName: \"kubernetes.io/projected/424659c2-e71b-4390-9eb3-1c8dce8df887-kube-api-access-gpmp9\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829240 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829197    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-tmp-dir\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829240 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829241    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-home\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829489 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829288    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-dshm\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829489 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829305    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-model-cache\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829489 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829328    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpmp9\" (UniqueName: \"kubernetes.io/projected/424659c2-e71b-4390-9eb3-1c8dce8df887-kube-api-access-gpmp9\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829489 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829361    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424659c2-e71b-4390-9eb3-1c8dce8df887-tls-certs\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829742 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829689    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-tmp-dir\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829917 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829743    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-model-cache\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.829917 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.829811    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-home\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.831808 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.831786    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-dshm\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.831939 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.831898    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424659c2-e71b-4390-9eb3-1c8dce8df887-tls-certs\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:47.837233 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:47.837215    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpmp9\" (UniqueName: \"kubernetes.io/projected/424659c2-e71b-4390-9eb3-1c8dce8df887-kube-api-access-gpmp9\") pod \"auth-test-model-7b406b-kserve-78c48fc847-x9cfr\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") " pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:48.018616 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:48.018585    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:48.156530 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:48.156501    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"]
Jul 09 20:21:48.158268 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:21:48.158245    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod424659c2_e71b_4390_9eb3_1c8dce8df887.slice/crio-20be750679cde689f53e4b50ee667132f1ab4423498252947c3d0acd92a4bbc9 WatchSource:0}: Error finding container 20be750679cde689f53e4b50ee667132f1ab4423498252947c3d0acd92a4bbc9: Status 404 returned error can't find the container with id 20be750679cde689f53e4b50ee667132f1ab4423498252947c3d0acd92a4bbc9
Jul 09 20:21:48.616174 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:48.616138    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr" event={"ID":"424659c2-e71b-4390-9eb3-1c8dce8df887","Type":"ContainerStarted","Data":"cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e"}
Jul 09 20:21:48.616174 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:48.616183    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr" event={"ID":"424659c2-e71b-4390-9eb3-1c8dce8df887","Type":"ContainerStarted","Data":"20be750679cde689f53e4b50ee667132f1ab4423498252947c3d0acd92a4bbc9"}
Jul 09 20:21:48.634401 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:48.634350    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr" podStartSLOduration=1.63433257 podStartE2EDuration="1.63433257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:21:48.63227859 +0000 UTC m=+2520.033413402" watchObservedRunningTime="2026-07-09 20:21:48.63433257 +0000 UTC m=+2520.035467379"
Jul 09 20:21:58.019629 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:58.019590    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:58.020155 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:58.019676    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:58.032845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:58.032812    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:21:58.687020 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:21:58.686982    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:22:13.697280 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.697242    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"]
Jul 09 20:22:13.701430 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.701409    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.704383 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.704360    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"kube-root-ca.crt\""
Jul 09 20:22:13.704383 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.704373    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"openshift-service-ca.crt\""
Jul 09 20:22:13.705491 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.705476    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"default-dockercfg-kr57t\""
Jul 09 20:22:13.705592 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.705541    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"auth-test-model-3452a7-kserve-self-signed-certs\""
Jul 09 20:22:13.717510 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.717464    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"]
Jul 09 20:22:13.794633 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.794585    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-model-cache\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.794633 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.794628    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8snls\" (UniqueName: \"kubernetes.io/projected/4a312688-8e92-468a-a798-4793e4579acb-kube-api-access-8snls\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.794911 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.794708    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-dshm\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.794911 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.794792    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-home\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.794911 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.794817    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a312688-8e92-468a-a798-4793e4579acb-tls-certs\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.794911 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.794856    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-tmp-dir\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.896356 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896310    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-dshm\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.896553 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896395    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-home\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.896553 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896421    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a312688-8e92-468a-a798-4793e4579acb-tls-certs\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.896553 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896464    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-tmp-dir\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.896553 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896523    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-model-cache\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.896553 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896549    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8snls\" (UniqueName: \"kubernetes.io/projected/4a312688-8e92-468a-a798-4793e4579acb-kube-api-access-8snls\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.896944 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896920    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-tmp-dir\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.897010 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896946    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-model-cache\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.897010 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.896926    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-home\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.898856 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.898830    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-dshm\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.899013 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.898997    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a312688-8e92-468a-a798-4793e4579acb-tls-certs\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:13.906865 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:13.906839    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8snls\" (UniqueName: \"kubernetes.io/projected/4a312688-8e92-468a-a798-4793e4579acb-kube-api-access-8snls\") pod \"auth-test-model-3452a7-kserve-77494685cf-x8fzv\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") " pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:14.011478 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:14.011395    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:14.154755 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:14.154726    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"]
Jul 09 20:22:14.156001 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:22:14.155973    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a312688_8e92_468a_a798_4793e4579acb.slice/crio-c9001cf489105f1830dbf1c0ccc88fedea7d534724ca1f7e39c60edb1a213dc6 WatchSource:0}: Error finding container c9001cf489105f1830dbf1c0ccc88fedea7d534724ca1f7e39c60edb1a213dc6: Status 404 returned error can't find the container with id c9001cf489105f1830dbf1c0ccc88fedea7d534724ca1f7e39c60edb1a213dc6
Jul 09 20:22:14.761318 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:14.761287    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv" event={"ID":"4a312688-8e92-468a-a798-4793e4579acb","Type":"ContainerStarted","Data":"67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd"}
Jul 09 20:22:14.761743 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:14.761326    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv" event={"ID":"4a312688-8e92-468a-a798-4793e4579acb","Type":"ContainerStarted","Data":"c9001cf489105f1830dbf1c0ccc88fedea7d534724ca1f7e39c60edb1a213dc6"}
Jul 09 20:22:14.779550 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:14.779501    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv" podStartSLOduration=1.779485135 podStartE2EDuration="1.779485135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:22:14.778763116 +0000 UTC m=+2546.179897924" watchObservedRunningTime="2026-07-09 20:22:14.779485135 +0000 UTC m=+2546.180619945"
Jul 09 20:22:24.012492 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:24.012453    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:24.012492 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:24.012501    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:24.025246 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:24.025216    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:22:24.832190 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:22:24.832163    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:24:25.079950 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:25.079921    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"]
Jul 09 20:24:26.009323 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:26.009291    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"]
Jul 09 20:24:40.100833 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.100717    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr" podUID="424659c2-e71b-4390-9eb3-1c8dce8df887" containerName="main" containerID="cri-o://cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e" gracePeriod=45
Jul 09 20:24:40.364146 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.364083    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:24:40.482036 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.481993    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424659c2-e71b-4390-9eb3-1c8dce8df887-tls-certs\") pod \"424659c2-e71b-4390-9eb3-1c8dce8df887\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") "
Jul 09 20:24:40.482036 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482035    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-home\") pod \"424659c2-e71b-4390-9eb3-1c8dce8df887\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") "
Jul 09 20:24:40.482398 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482256    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpmp9\" (UniqueName: \"kubernetes.io/projected/424659c2-e71b-4390-9eb3-1c8dce8df887-kube-api-access-gpmp9\") pod \"424659c2-e71b-4390-9eb3-1c8dce8df887\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") "
Jul 09 20:24:40.482398 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482301    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-home" (OuterVolumeSpecName: "home") pod "424659c2-e71b-4390-9eb3-1c8dce8df887" (UID: "424659c2-e71b-4390-9eb3-1c8dce8df887"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:40.482398 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482327    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-tmp-dir\") pod \"424659c2-e71b-4390-9eb3-1c8dce8df887\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") "
Jul 09 20:24:40.482398 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482356    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-dshm\") pod \"424659c2-e71b-4390-9eb3-1c8dce8df887\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") "
Jul 09 20:24:40.482613 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482412    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-model-cache\") pod \"424659c2-e71b-4390-9eb3-1c8dce8df887\" (UID: \"424659c2-e71b-4390-9eb3-1c8dce8df887\") "
Jul 09 20:24:40.482704 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482606    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "424659c2-e71b-4390-9eb3-1c8dce8df887" (UID: "424659c2-e71b-4390-9eb3-1c8dce8df887"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:40.482936 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482894    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-model-cache" (OuterVolumeSpecName: "model-cache") pod "424659c2-e71b-4390-9eb3-1c8dce8df887" (UID: "424659c2-e71b-4390-9eb3-1c8dce8df887"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:40.482936 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482904    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:40.482936 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.482923    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:40.484900 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.484866    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-dshm" (OuterVolumeSpecName: "dshm") pod "424659c2-e71b-4390-9eb3-1c8dce8df887" (UID: "424659c2-e71b-4390-9eb3-1c8dce8df887"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:40.485008 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.484971    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/424659c2-e71b-4390-9eb3-1c8dce8df887-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "424659c2-e71b-4390-9eb3-1c8dce8df887" (UID: "424659c2-e71b-4390-9eb3-1c8dce8df887"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:24:40.485008 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.484994    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/424659c2-e71b-4390-9eb3-1c8dce8df887-kube-api-access-gpmp9" (OuterVolumeSpecName: "kube-api-access-gpmp9") pod "424659c2-e71b-4390-9eb3-1c8dce8df887" (UID: "424659c2-e71b-4390-9eb3-1c8dce8df887"). InnerVolumeSpecName "kube-api-access-gpmp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:24:40.550985 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.550951    2527 generic.go:356] "Generic (PLEG): container finished" podID="424659c2-e71b-4390-9eb3-1c8dce8df887" containerID="cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e" exitCode=0
Jul 09 20:24:40.551150 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.551001    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"
Jul 09 20:24:40.551150 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.551045    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr" event={"ID":"424659c2-e71b-4390-9eb3-1c8dce8df887","Type":"ContainerDied","Data":"cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e"}
Jul 09 20:24:40.551150 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.551093    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr" event={"ID":"424659c2-e71b-4390-9eb3-1c8dce8df887","Type":"ContainerDied","Data":"20be750679cde689f53e4b50ee667132f1ab4423498252947c3d0acd92a4bbc9"}
Jul 09 20:24:40.551150 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.551116    2527 scope.go:117] "RemoveContainer" containerID="cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e"
Jul 09 20:24:40.560922 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.560906    2527 scope.go:117] "RemoveContainer" containerID="cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e"
Jul 09 20:24:40.561192 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:24:40.561171    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e\": container with ID starting with cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e not found: ID does not exist" containerID="cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e"
Jul 09 20:24:40.561257 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.561204    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e"} err="failed to get container status \"cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e\": rpc error: code = NotFound desc = could not find container \"cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e\": container with ID starting with cfddc4937c76667d5557a375d648979eea5f1a614e5e8285fce0dff9702de30e not found: ID does not exist"
Jul 09 20:24:40.576487 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.576458    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"]
Jul 09 20:24:40.583008 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.582979    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/auth-test-model-7b406b-kserve-78c48fc847-x9cfr"]
Jul 09 20:24:40.583526 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.583506    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424659c2-e71b-4390-9eb3-1c8dce8df887-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:40.583580 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.583533    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpmp9\" (UniqueName: \"kubernetes.io/projected/424659c2-e71b-4390-9eb3-1c8dce8df887-kube-api-access-gpmp9\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:40.583580 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.583547    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:40.583580 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:40.583562    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424659c2-e71b-4390-9eb3-1c8dce8df887-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:41.047727 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.047670    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv" podUID="4a312688-8e92-468a-a798-4793e4579acb" containerName="main" containerID="cri-o://67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd" gracePeriod=45
Jul 09 20:24:41.077712 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.077684    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424659c2-e71b-4390-9eb3-1c8dce8df887" path="/var/lib/kubelet/pods/424659c2-e71b-4390-9eb3-1c8dce8df887/volumes"
Jul 09 20:24:41.309562 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.309499    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:24:41.389559 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.389526    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-tmp-dir\") pod \"4a312688-8e92-468a-a798-4793e4579acb\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") "
Jul 09 20:24:41.389797 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.389619    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8snls\" (UniqueName: \"kubernetes.io/projected/4a312688-8e92-468a-a798-4793e4579acb-kube-api-access-8snls\") pod \"4a312688-8e92-468a-a798-4793e4579acb\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") "
Jul 09 20:24:41.389797 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.389689    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-dshm\") pod \"4a312688-8e92-468a-a798-4793e4579acb\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") "
Jul 09 20:24:41.389797 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.389710    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-home\") pod \"4a312688-8e92-468a-a798-4793e4579acb\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") "
Jul 09 20:24:41.389797 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.389739    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a312688-8e92-468a-a798-4793e4579acb-tls-certs\") pod \"4a312688-8e92-468a-a798-4793e4579acb\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") "
Jul 09 20:24:41.389797 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.389775    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-model-cache\") pod \"4a312688-8e92-468a-a798-4793e4579acb\" (UID: \"4a312688-8e92-468a-a798-4793e4579acb\") "
Jul 09 20:24:41.390046 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.389881    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a312688-8e92-468a-a798-4793e4579acb" (UID: "4a312688-8e92-468a-a798-4793e4579acb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:41.390100 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.390067    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a312688-8e92-468a-a798-4793e4579acb" (UID: "4a312688-8e92-468a-a798-4793e4579acb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:41.390152 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.390105    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-home" (OuterVolumeSpecName: "home") pod "4a312688-8e92-468a-a798-4793e4579acb" (UID: "4a312688-8e92-468a-a798-4793e4579acb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:41.390152 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.390136    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:41.392178 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.392148    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a312688-8e92-468a-a798-4793e4579acb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a312688-8e92-468a-a798-4793e4579acb" (UID: "4a312688-8e92-468a-a798-4793e4579acb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:24:41.392302 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.392281    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-dshm" (OuterVolumeSpecName: "dshm") pod "4a312688-8e92-468a-a798-4793e4579acb" (UID: "4a312688-8e92-468a-a798-4793e4579acb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:24:41.392362 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.392305    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a312688-8e92-468a-a798-4793e4579acb-kube-api-access-8snls" (OuterVolumeSpecName: "kube-api-access-8snls") pod "4a312688-8e92-468a-a798-4793e4579acb" (UID: "4a312688-8e92-468a-a798-4793e4579acb"). InnerVolumeSpecName "kube-api-access-8snls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:24:41.491206 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.491170    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8snls\" (UniqueName: \"kubernetes.io/projected/4a312688-8e92-468a-a798-4793e4579acb-kube-api-access-8snls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:41.491206 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.491199    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:41.491206 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.491211    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:41.491439 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.491220    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a312688-8e92-468a-a798-4793e4579acb-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:41.491439 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.491233    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a312688-8e92-468a-a798-4793e4579acb-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:24:41.558731 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.558699    2527 generic.go:356] "Generic (PLEG): container finished" podID="4a312688-8e92-468a-a798-4793e4579acb" containerID="67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd" exitCode=0
Jul 09 20:24:41.558918 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.558739    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"
Jul 09 20:24:41.558918 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.558783    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv" event={"ID":"4a312688-8e92-468a-a798-4793e4579acb","Type":"ContainerDied","Data":"67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd"}
Jul 09 20:24:41.558918 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.558823    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv" event={"ID":"4a312688-8e92-468a-a798-4793e4579acb","Type":"ContainerDied","Data":"c9001cf489105f1830dbf1c0ccc88fedea7d534724ca1f7e39c60edb1a213dc6"}
Jul 09 20:24:41.558918 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.558840    2527 scope.go:117] "RemoveContainer" containerID="67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd"
Jul 09 20:24:41.568841 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.568823    2527 scope.go:117] "RemoveContainer" containerID="67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd"
Jul 09 20:24:41.569142 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:24:41.569119    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd\": container with ID starting with 67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd not found: ID does not exist" containerID="67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd"
Jul 09 20:24:41.569199 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.569150    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd"} err="failed to get container status \"67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd\": rpc error: code = NotFound desc = could not find container \"67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd\": container with ID starting with 67fb9ce02b147dc54b44435060d0a04ba2684fab6cc72058971134dd3ad014bd not found: ID does not exist"
Jul 09 20:24:41.582336 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.582302    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"]
Jul 09 20:24:41.585608 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:41.585587    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/auth-test-model-3452a7-kserve-77494685cf-x8fzv"]
Jul 09 20:24:43.083061 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:24:43.083025    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a312688-8e92-468a-a798-4793e4579acb" path="/var/lib/kubelet/pods/4a312688-8e92-468a-a798-4793e4579acb/volumes"
Jul 09 20:25:13.998853 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:13.998816    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"]
Jul 09 20:25:13.999453 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:13.999419    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424659c2-e71b-4390-9eb3-1c8dce8df887" containerName="main"
Jul 09 20:25:13.999453 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:13.999438    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="424659c2-e71b-4390-9eb3-1c8dce8df887" containerName="main"
Jul 09 20:25:13.999572 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:13.999480    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a312688-8e92-468a-a798-4793e4579acb" containerName="main"
Jul 09 20:25:13.999572 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:13.999490    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a312688-8e92-468a-a798-4793e4579acb" containerName="main"
Jul 09 20:25:13.999711 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:13.999577    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a312688-8e92-468a-a798-4793e4579acb" containerName="main"
Jul 09 20:25:13.999711 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:13.999597    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="424659c2-e71b-4390-9eb3-1c8dce8df887" containerName="main"
Jul 09 20:25:14.002508 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.002488    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.006689 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.006666    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"sub-test-model-3452a7-kserve-self-signed-certs\""
Jul 09 20:25:14.006689 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.006680    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"kube-root-ca.crt\""
Jul 09 20:25:14.006888 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.006697    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"openshift-service-ca.crt\""
Jul 09 20:25:14.006888 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.006671    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"default-dockercfg-kr57t\""
Jul 09 20:25:14.011458 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.011434    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"]
Jul 09 20:25:14.110905 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.110871    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-home\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.111084 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.110923    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tmp-dir\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.111084 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.110989    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpvzh\" (UniqueName: \"kubernetes.io/projected/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-kube-api-access-hpvzh\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.111084 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.111051    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-dshm\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.111198 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.111102    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-model-cache\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.111198 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.111157    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tls-certs\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.212476 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.212437    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-model-cache\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.212710 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.212497    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tls-certs\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.212710 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.212555    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-home\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.212710 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.212609    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tmp-dir\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.212710 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.212689    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpvzh\" (UniqueName: \"kubernetes.io/projected/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-kube-api-access-hpvzh\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.212942 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.212748    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-dshm\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.213002 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.212938    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-model-cache\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.213054 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.213013    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tmp-dir\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.213108 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.213058    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-home\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.215070 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.215049    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-dshm\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.215390 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.215368    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tls-certs\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.221351 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.221325    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpvzh\" (UniqueName: \"kubernetes.io/projected/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-kube-api-access-hpvzh\") pod \"sub-test-model-3452a7-kserve-58bf5f44dc-8p525\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") " pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.313710 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.313631    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:14.454631 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.454605    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"]
Jul 09 20:25:14.459731 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:25:14.459138    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfad2d53c_1d86_4a4d_bbd7_c8cfb6834dd6.slice/crio-91f4854310fc20faf2412774a3e19bcad8bced63d17c0bd961347ab0bfd14507 WatchSource:0}: Error finding container 91f4854310fc20faf2412774a3e19bcad8bced63d17c0bd961347ab0bfd14507: Status 404 returned error can't find the container with id 91f4854310fc20faf2412774a3e19bcad8bced63d17c0bd961347ab0bfd14507
Jul 09 20:25:14.744010 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.743902    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525" event={"ID":"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6","Type":"ContainerStarted","Data":"2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc"}
Jul 09 20:25:14.744010 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.743957    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525" event={"ID":"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6","Type":"ContainerStarted","Data":"91f4854310fc20faf2412774a3e19bcad8bced63d17c0bd961347ab0bfd14507"}
Jul 09 20:25:14.763949 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:14.763893    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525" podStartSLOduration=1.763873252 podStartE2EDuration="1.763873252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:25:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:25:14.761848471 +0000 UTC m=+2726.162983284" watchObservedRunningTime="2026-07-09 20:25:14.763873252 +0000 UTC m=+2726.165008062"
Jul 09 20:25:24.314855 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:24.314816    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:24.315281 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:24.314871    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:24.327886 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:24.327857    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:24.808289 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:24.808257    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:25:55.410363 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:25:55.410322    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"]
Jul 09 20:26:10.431618 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.431577    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525" podUID="fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" containerName="main" containerID="cri-o://2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc" gracePeriod=45
Jul 09 20:26:10.674604 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.674579    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:26:10.756340 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756239    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpvzh\" (UniqueName: \"kubernetes.io/projected/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-kube-api-access-hpvzh\") pod \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") "
Jul 09 20:26:10.756496 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756357    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-home\") pod \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") "
Jul 09 20:26:10.756496 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756426    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-dshm\") pod \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") "
Jul 09 20:26:10.756496 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756451    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tmp-dir\") pod \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") "
Jul 09 20:26:10.756496 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756481    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-model-cache\") pod \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") "
Jul 09 20:26:10.756690 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756506    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-home" (OuterVolumeSpecName: "home") pod "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" (UID: "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:26:10.756690 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756515    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tls-certs\") pod \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\" (UID: \"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6\") "
Jul 09 20:26:10.756810 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756715    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-model-cache" (OuterVolumeSpecName: "model-cache") pod "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" (UID: "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:26:10.756810 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.756773    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" (UID: "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:26:10.757144 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.757022    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:26:10.757259 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.757168    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:26:10.757259 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.757182    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:26:10.758559 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.758540    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-dshm" (OuterVolumeSpecName: "dshm") pod "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" (UID: "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:26:10.758691 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.758671    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" (UID: "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:26:10.758740 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.758715    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-kube-api-access-hpvzh" (OuterVolumeSpecName: "kube-api-access-hpvzh") pod "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" (UID: "fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6"). InnerVolumeSpecName "kube-api-access-hpvzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:26:10.858750 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.858700    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpvzh\" (UniqueName: \"kubernetes.io/projected/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-kube-api-access-hpvzh\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:26:10.858750 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.858744    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:26:10.858750 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:10.858758    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:26:11.045936 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.045899    2527 generic.go:356] "Generic (PLEG): container finished" podID="fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" containerID="2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc" exitCode=0
Jul 09 20:26:11.046129 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.045959    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"
Jul 09 20:26:11.046129 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.045990    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525" event={"ID":"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6","Type":"ContainerDied","Data":"2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc"}
Jul 09 20:26:11.046129 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.046039    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525" event={"ID":"fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6","Type":"ContainerDied","Data":"91f4854310fc20faf2412774a3e19bcad8bced63d17c0bd961347ab0bfd14507"}
Jul 09 20:26:11.046129 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.046056    2527 scope.go:117] "RemoveContainer" containerID="2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc"
Jul 09 20:26:11.055784 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.055767    2527 scope.go:117] "RemoveContainer" containerID="2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc"
Jul 09 20:26:11.056066 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:26:11.056047    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc\": container with ID starting with 2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc not found: ID does not exist" containerID="2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc"
Jul 09 20:26:11.056105 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.056075    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc"} err="failed to get container status \"2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc\": rpc error: code = NotFound desc = could not find container \"2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc\": container with ID starting with 2c9fb7c48a78be3963bc9bd87907e97015376654e51b7caaf7443a39cbb9c3bc not found: ID does not exist"
Jul 09 20:26:11.067557 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.067530    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"]
Jul 09 20:26:11.071143 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.071121    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/sub-test-model-3452a7-kserve-58bf5f44dc-8p525"]
Jul 09 20:26:11.077585 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:11.077562    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" path="/var/lib/kubelet/pods/fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6/volumes"
Jul 09 20:26:21.293131 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.293085    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"]
Jul 09 20:26:21.293593 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.293529    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" containerName="main"
Jul 09 20:26:21.293593 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.293540    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" containerName="main"
Jul 09 20:26:21.293699 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.293616    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad2d53c-1d86-4a4d-bbd7-c8cfb6834dd6" containerName="main"
Jul 09 20:26:21.298001 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.297983    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.302038 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.302011    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"openshift-service-ca.crt\""
Jul 09 20:26:21.302194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.302077    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"default-dockercfg-mlp92\""
Jul 09 20:26:21.302194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.302134    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"rate-test-model-7b406b-kserve-self-signed-certs\""
Jul 09 20:26:21.302194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.302086    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b406b\"/\"kube-root-ca.crt\""
Jul 09 20:26:21.304873 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.304845    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"]
Jul 09 20:26:21.359687 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.359620    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-model-cache\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.359860 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.359735    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-home\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.359860 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.359786    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tmp-dir\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.359860 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.359850    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-dshm\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.360016 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.359975    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hsrg\" (UniqueName: \"kubernetes.io/projected/4ff3e041-9ef9-42da-aa4e-4baaee671b57-kube-api-access-9hsrg\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.360061 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.360042    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tls-certs\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461238 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461200    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hsrg\" (UniqueName: \"kubernetes.io/projected/4ff3e041-9ef9-42da-aa4e-4baaee671b57-kube-api-access-9hsrg\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461471 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461264    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tls-certs\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461471 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461313    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-model-cache\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461471 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461360    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-home\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461632 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461488    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tmp-dir\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461632 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461537    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-dshm\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461834 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461812    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-model-cache\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461902 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461857    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-home\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.461942 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.461901    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tmp-dir\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.463815 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.463787    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-dshm\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.464052 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.464035    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tls-certs\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.470726 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.470694    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hsrg\" (UniqueName: \"kubernetes.io/projected/4ff3e041-9ef9-42da-aa4e-4baaee671b57-kube-api-access-9hsrg\") pod \"rate-test-model-7b406b-kserve-55695b47c5-f22nm\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") " pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.608302 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.608211    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:21.751844 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:21.751807    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"]
Jul 09 20:26:21.755128 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:26:21.755100    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ff3e041_9ef9_42da_aa4e_4baaee671b57.slice/crio-694079d3a54a74582d96378b79d70e32e13ef3d2465cf6e0d8fd91701d3c7d05 WatchSource:0}: Error finding container 694079d3a54a74582d96378b79d70e32e13ef3d2465cf6e0d8fd91701d3c7d05: Status 404 returned error can't find the container with id 694079d3a54a74582d96378b79d70e32e13ef3d2465cf6e0d8fd91701d3c7d05
Jul 09 20:26:22.106574 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:22.106535    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm" event={"ID":"4ff3e041-9ef9-42da-aa4e-4baaee671b57","Type":"ContainerStarted","Data":"c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e"}
Jul 09 20:26:22.106801 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:22.106586    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm" event={"ID":"4ff3e041-9ef9-42da-aa4e-4baaee671b57","Type":"ContainerStarted","Data":"694079d3a54a74582d96378b79d70e32e13ef3d2465cf6e0d8fd91701d3c7d05"}
Jul 09 20:26:22.126370 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:22.126312    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm" podStartSLOduration=1.126296786 podStartE2EDuration="1.126296786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:26:22.123955692 +0000 UTC m=+2793.525090493" watchObservedRunningTime="2026-07-09 20:26:22.126296786 +0000 UTC m=+2793.527431596"
Jul 09 20:26:31.609386 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:31.609349    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:31.610022 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:31.609410    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:31.621922 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:31.621893    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:32.177940 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:32.177902    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:26:48.886214 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.886179    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"]
Jul 09 20:26:48.889909 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.889892    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:48.892571 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.892547    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"kube-root-ca.crt\""
Jul 09 20:26:48.893740 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.893712    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"default-dockercfg-kr57t\""
Jul 09 20:26:48.893740 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.893731    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"rate-test-model-3452a7-kserve-self-signed-certs\""
Jul 09 20:26:48.893916 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.893770    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"openshift-service-ca.crt\""
Jul 09 20:26:48.903313 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.903279    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"]
Jul 09 20:26:48.941812 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.941780    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba01b44b-e2d4-4643-8494-34335324d87b-tls-certs\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:48.941992 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.941857    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-dshm\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:48.941992 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.941884    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-model-cache\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:48.941992 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.941900    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwrc9\" (UniqueName: \"kubernetes.io/projected/ba01b44b-e2d4-4643-8494-34335324d87b-kube-api-access-mwrc9\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:48.941992 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.941936    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-tmp-dir\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:48.942133 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:48.942054    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-home\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.042722 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.042680    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-tmp-dir\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.042896 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.042762    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-home\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.042896 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.042786    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba01b44b-e2d4-4643-8494-34335324d87b-tls-certs\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.042987 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.042938    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-dshm\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.043044 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.042990    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-model-cache\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.043044 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.043021    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwrc9\" (UniqueName: \"kubernetes.io/projected/ba01b44b-e2d4-4643-8494-34335324d87b-kube-api-access-mwrc9\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.043153 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.043066    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-tmp-dir\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.043208 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.043158    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-home\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.043274 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.043255    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-model-cache\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.045216 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.045195    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-dshm\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.045435 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.045419    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"rate-test-model-3452a7-kserve-self-signed-certs\""
Jul 09 20:26:49.050228 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.050206    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"kube-root-ca.crt\""
Jul 09 20:26:49.055625 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.055605    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba01b44b-e2d4-4643-8494-34335324d87b-tls-certs\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.060579 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.060558    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"openshift-service-ca.crt\""
Jul 09 20:26:49.071076 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.071051    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwrc9\" (UniqueName: \"kubernetes.io/projected/ba01b44b-e2d4-4643-8494-34335324d87b-kube-api-access-mwrc9\") pod \"rate-test-model-3452a7-kserve-578594bb75-ngtpk\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") " pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.207855 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.207772    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3452a7\"/\"default-dockercfg-kr57t\""
Jul 09 20:26:49.215169 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.215144    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:49.359622 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.359593    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"]
Jul 09 20:26:49.361091 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:26:49.361069    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba01b44b_e2d4_4643_8494_34335324d87b.slice/crio-c133eadb535bb7ed5cc9b44bfdc4cce4c37d2b0a2cb38b5bcb7376863e882a1e WatchSource:0}: Error finding container c133eadb535bb7ed5cc9b44bfdc4cce4c37d2b0a2cb38b5bcb7376863e882a1e: Status 404 returned error can't find the container with id c133eadb535bb7ed5cc9b44bfdc4cce4c37d2b0a2cb38b5bcb7376863e882a1e
Jul 09 20:26:49.365453 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:49.365434    2527 scope.go:117] "RemoveContainer" containerID="92db7452a5a7e77a2430d35bfaa64d47735bf99c3b46387c957cab1d379d3c8e"
Jul 09 20:26:50.266489 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:50.266452    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk" event={"ID":"ba01b44b-e2d4-4643-8494-34335324d87b","Type":"ContainerStarted","Data":"317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6"}
Jul 09 20:26:50.266489 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:50.266493    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk" event={"ID":"ba01b44b-e2d4-4643-8494-34335324d87b","Type":"ContainerStarted","Data":"c133eadb535bb7ed5cc9b44bfdc4cce4c37d2b0a2cb38b5bcb7376863e882a1e"}
Jul 09 20:26:50.288244 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:50.288193    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk" podStartSLOduration=2.288177893 podStartE2EDuration="2.288177893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:26:50.286616951 +0000 UTC m=+2821.687751774" watchObservedRunningTime="2026-07-09 20:26:50.288177893 +0000 UTC m=+2821.689312757"
Jul 09 20:26:59.216389 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:59.216353    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:59.216811 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:59.216408    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:59.228508 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:59.228477    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:26:59.337747 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:26:59.337713    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:27:30.295241 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:30.295206    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"]
Jul 09 20:27:31.177228 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:31.177194    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"]
Jul 09 20:27:45.318472 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.318351    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm" podUID="4ff3e041-9ef9-42da-aa4e-4baaee671b57" containerName="main" containerID="cri-o://c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e" gracePeriod=45
Jul 09 20:27:45.569354 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.569285    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:27:45.575298 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.575270    2527 generic.go:356] "Generic (PLEG): container finished" podID="4ff3e041-9ef9-42da-aa4e-4baaee671b57" containerID="c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e" exitCode=0
Jul 09 20:27:45.575457 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.575315    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"
Jul 09 20:27:45.575457 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.575347    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm" event={"ID":"4ff3e041-9ef9-42da-aa4e-4baaee671b57","Type":"ContainerDied","Data":"c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e"}
Jul 09 20:27:45.575457 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.575394    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm" event={"ID":"4ff3e041-9ef9-42da-aa4e-4baaee671b57","Type":"ContainerDied","Data":"694079d3a54a74582d96378b79d70e32e13ef3d2465cf6e0d8fd91701d3c7d05"}
Jul 09 20:27:45.575457 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.575419    2527 scope.go:117] "RemoveContainer" containerID="c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e"
Jul 09 20:27:45.585730 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.585707    2527 scope.go:117] "RemoveContainer" containerID="c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e"
Jul 09 20:27:45.586056 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:27:45.586029    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e\": container with ID starting with c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e not found: ID does not exist" containerID="c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e"
Jul 09 20:27:45.586205 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.586066    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e"} err="failed to get container status \"c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e\": rpc error: code = NotFound desc = could not find container \"c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e\": container with ID starting with c6a9d6fb576bc7cc3fd98a8a04fea98aff543af51e729790ee6966419d1a6d8e not found: ID does not exist"
Jul 09 20:27:45.620395 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620359    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-dshm\") pod \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") "
Jul 09 20:27:45.620395 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620398    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-home\") pod \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") "
Jul 09 20:27:45.620592 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620433    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hsrg\" (UniqueName: \"kubernetes.io/projected/4ff3e041-9ef9-42da-aa4e-4baaee671b57-kube-api-access-9hsrg\") pod \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") "
Jul 09 20:27:45.620592 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620499    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tmp-dir\") pod \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") "
Jul 09 20:27:45.620592 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620560    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-model-cache\") pod \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") "
Jul 09 20:27:45.620592 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620586    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tls-certs\") pod \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\" (UID: \"4ff3e041-9ef9-42da-aa4e-4baaee671b57\") "
Jul 09 20:27:45.620841 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620745    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-home" (OuterVolumeSpecName: "home") pod "4ff3e041-9ef9-42da-aa4e-4baaee671b57" (UID: "4ff3e041-9ef9-42da-aa4e-4baaee671b57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:45.620841 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620830    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4ff3e041-9ef9-42da-aa4e-4baaee671b57" (UID: "4ff3e041-9ef9-42da-aa4e-4baaee671b57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:45.620943 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.620847    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-model-cache" (OuterVolumeSpecName: "model-cache") pod "4ff3e041-9ef9-42da-aa4e-4baaee671b57" (UID: "4ff3e041-9ef9-42da-aa4e-4baaee671b57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:45.621078 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.621059    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:45.621147 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.621090    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:45.621147 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.621105    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:45.622754 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.622734    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-dshm" (OuterVolumeSpecName: "dshm") pod "4ff3e041-9ef9-42da-aa4e-4baaee671b57" (UID: "4ff3e041-9ef9-42da-aa4e-4baaee671b57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:45.622843 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.622736    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff3e041-9ef9-42da-aa4e-4baaee671b57-kube-api-access-9hsrg" (OuterVolumeSpecName: "kube-api-access-9hsrg") pod "4ff3e041-9ef9-42da-aa4e-4baaee671b57" (UID: "4ff3e041-9ef9-42da-aa4e-4baaee671b57"). InnerVolumeSpecName "kube-api-access-9hsrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:27:45.622843 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.622769    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4ff3e041-9ef9-42da-aa4e-4baaee671b57" (UID: "4ff3e041-9ef9-42da-aa4e-4baaee671b57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:27:45.722178 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.722142    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff3e041-9ef9-42da-aa4e-4baaee671b57-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:45.722178 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.722173    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hsrg\" (UniqueName: \"kubernetes.io/projected/4ff3e041-9ef9-42da-aa4e-4baaee671b57-kube-api-access-9hsrg\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:45.722178 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.722185    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff3e041-9ef9-42da-aa4e-4baaee671b57-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:45.898495 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.898462    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"]
Jul 09 20:27:45.901436 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:45.901409    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b406b/rate-test-model-7b406b-kserve-55695b47c5-f22nm"]
Jul 09 20:27:46.199336 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.199250    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk" podUID="ba01b44b-e2d4-4643-8494-34335324d87b" containerName="main" containerID="cri-o://317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6" gracePeriod=45
Jul 09 20:27:46.462194 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.462134    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:27:46.529586 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529552    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-model-cache\") pod \"ba01b44b-e2d4-4643-8494-34335324d87b\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") "
Jul 09 20:27:46.529816 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529608    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-dshm\") pod \"ba01b44b-e2d4-4643-8494-34335324d87b\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") "
Jul 09 20:27:46.529816 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529704    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-tmp-dir\") pod \"ba01b44b-e2d4-4643-8494-34335324d87b\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") "
Jul 09 20:27:46.529816 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529779    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba01b44b-e2d4-4643-8494-34335324d87b-tls-certs\") pod \"ba01b44b-e2d4-4643-8494-34335324d87b\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") "
Jul 09 20:27:46.529989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529828    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-home\") pod \"ba01b44b-e2d4-4643-8494-34335324d87b\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") "
Jul 09 20:27:46.529989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529856    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba01b44b-e2d4-4643-8494-34335324d87b" (UID: "ba01b44b-e2d4-4643-8494-34335324d87b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:46.529989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529865    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwrc9\" (UniqueName: \"kubernetes.io/projected/ba01b44b-e2d4-4643-8494-34335324d87b-kube-api-access-mwrc9\") pod \"ba01b44b-e2d4-4643-8494-34335324d87b\" (UID: \"ba01b44b-e2d4-4643-8494-34335324d87b\") "
Jul 09 20:27:46.529989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.529948    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba01b44b-e2d4-4643-8494-34335324d87b" (UID: "ba01b44b-e2d4-4643-8494-34335324d87b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:46.530180 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.530014    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-home" (OuterVolumeSpecName: "home") pod "ba01b44b-e2d4-4643-8494-34335324d87b" (UID: "ba01b44b-e2d4-4643-8494-34335324d87b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:46.530267 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.530242    2527 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-tmp-dir\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:46.530369 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.530313    2527 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-home\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:46.530461 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.530424    2527 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-model-cache\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:46.531973 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.531944    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba01b44b-e2d4-4643-8494-34335324d87b-kube-api-access-mwrc9" (OuterVolumeSpecName: "kube-api-access-mwrc9") pod "ba01b44b-e2d4-4643-8494-34335324d87b" (UID: "ba01b44b-e2d4-4643-8494-34335324d87b"). InnerVolumeSpecName "kube-api-access-mwrc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:27:46.531973 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.531952    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-dshm" (OuterVolumeSpecName: "dshm") pod "ba01b44b-e2d4-4643-8494-34335324d87b" (UID: "ba01b44b-e2d4-4643-8494-34335324d87b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 20:27:46.532089 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.531994    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba01b44b-e2d4-4643-8494-34335324d87b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba01b44b-e2d4-4643-8494-34335324d87b" (UID: "ba01b44b-e2d4-4643-8494-34335324d87b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:27:46.583069 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.583033    2527 generic.go:356] "Generic (PLEG): container finished" podID="ba01b44b-e2d4-4643-8494-34335324d87b" containerID="317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6" exitCode=0
Jul 09 20:27:46.583238 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.583087    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"
Jul 09 20:27:46.583238 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.583115    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk" event={"ID":"ba01b44b-e2d4-4643-8494-34335324d87b","Type":"ContainerDied","Data":"317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6"}
Jul 09 20:27:46.583238 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.583162    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk" event={"ID":"ba01b44b-e2d4-4643-8494-34335324d87b","Type":"ContainerDied","Data":"c133eadb535bb7ed5cc9b44bfdc4cce4c37d2b0a2cb38b5bcb7376863e882a1e"}
Jul 09 20:27:46.583238 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.583182    2527 scope.go:117] "RemoveContainer" containerID="317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6"
Jul 09 20:27:46.592546 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.592516    2527 scope.go:117] "RemoveContainer" containerID="317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6"
Jul 09 20:27:46.592862 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:27:46.592830    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6\": container with ID starting with 317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6 not found: ID does not exist" containerID="317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6"
Jul 09 20:27:46.592963 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.592865    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6"} err="failed to get container status \"317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6\": rpc error: code = NotFound desc = could not find container \"317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6\": container with ID starting with 317b2ea04d16bb5c1b326f37e583824f1dccd7c51e438e29a199ea82904688d6 not found: ID does not exist"
Jul 09 20:27:46.607695 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.607663    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"]
Jul 09 20:27:46.611844 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.611823    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3452a7/rate-test-model-3452a7-kserve-578594bb75-ngtpk"]
Jul 09 20:27:46.631284 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.631258    2527 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba01b44b-e2d4-4643-8494-34335324d87b-dshm\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:46.631284 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.631282    2527 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba01b44b-e2d4-4643-8494-34335324d87b-tls-certs\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:46.631431 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:46.631291    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwrc9\" (UniqueName: \"kubernetes.io/projected/ba01b44b-e2d4-4643-8494-34335324d87b-kube-api-access-mwrc9\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:47.077790 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.077750    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ff3e041-9ef9-42da-aa4e-4baaee671b57" path="/var/lib/kubelet/pods/4ff3e041-9ef9-42da-aa4e-4baaee671b57/volumes"
Jul 09 20:27:47.078271 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.078250    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba01b44b-e2d4-4643-8494-34335324d87b" path="/var/lib/kubelet/pods/ba01b44b-e2d4-4643-8494-34335324d87b/volumes"
Jul 09 20:27:47.628428 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.628401    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7607-140230641700272"]
Jul 09 20:27:47.628886 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.628823    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ff3e041-9ef9-42da-aa4e-4baaee671b57" containerName="main"
Jul 09 20:27:47.628886 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.628836    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ff3e041-9ef9-42da-aa4e-4baaee671b57" containerName="main"
Jul 09 20:27:47.628886 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.628847    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba01b44b-e2d4-4643-8494-34335324d87b" containerName="main"
Jul 09 20:27:47.628886 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.628852    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba01b44b-e2d4-4643-8494-34335324d87b" containerName="main"
Jul 09 20:27:47.629064 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.628916    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba01b44b-e2d4-4643-8494-34335324d87b" containerName="main"
Jul 09 20:27:47.629064 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.628926    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ff3e041-9ef9-42da-aa4e-4baaee671b57" containerName="main"
Jul 09 20:27:47.633300 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.633282    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7607-140230641700272"
Jul 09 20:27:47.636351 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.636331    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-qvtcl\""
Jul 09 20:27:47.640962 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.640943    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7607-140230641700272"]
Jul 09 20:27:47.741952 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.741917    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x9gz\" (UniqueName: \"kubernetes.io/projected/0ac009f7-e845-437c-a247-19a858235f97-kube-api-access-4x9gz\") pod \"test-curl-7607-140230641700272\" (UID: \"0ac009f7-e845-437c-a247-19a858235f97\") " pod="openshift-ingress/test-curl-7607-140230641700272"
Jul 09 20:27:47.843089 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.843052    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x9gz\" (UniqueName: \"kubernetes.io/projected/0ac009f7-e845-437c-a247-19a858235f97-kube-api-access-4x9gz\") pod \"test-curl-7607-140230641700272\" (UID: \"0ac009f7-e845-437c-a247-19a858235f97\") " pod="openshift-ingress/test-curl-7607-140230641700272"
Jul 09 20:27:47.851247 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.851215    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x9gz\" (UniqueName: \"kubernetes.io/projected/0ac009f7-e845-437c-a247-19a858235f97-kube-api-access-4x9gz\") pod \"test-curl-7607-140230641700272\" (UID: \"0ac009f7-e845-437c-a247-19a858235f97\") " pod="openshift-ingress/test-curl-7607-140230641700272"
Jul 09 20:27:47.944173 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:47.944083    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7607-140230641700272"
Jul 09 20:27:48.087480 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:48.087448    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7607-140230641700272"]
Jul 09 20:27:48.089564 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:27:48.089508    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ac009f7_e845_437c_a247_19a858235f97.slice/crio-bb93f571848f98bf6148286a7f7e95c77d72e89c1fd5391943804f53cb7f5f64 WatchSource:0}: Error finding container bb93f571848f98bf6148286a7f7e95c77d72e89c1fd5391943804f53cb7f5f64: Status 404 returned error can't find the container with id bb93f571848f98bf6148286a7f7e95c77d72e89c1fd5391943804f53cb7f5f64
Jul 09 20:27:48.093888 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:48.093864    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 20:27:48.597161 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:48.597125    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7607-140230641700272" event={"ID":"0ac009f7-e845-437c-a247-19a858235f97","Type":"ContainerStarted","Data":"bb93f571848f98bf6148286a7f7e95c77d72e89c1fd5391943804f53cb7f5f64"}
Jul 09 20:27:49.603502 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:49.603461    2527 generic.go:356] "Generic (PLEG): container finished" podID="0ac009f7-e845-437c-a247-19a858235f97" containerID="0b513373fa641f4a0136ecdfe3195a972ca5d8853619c50aa7757d25039c3907" exitCode=0
Jul 09 20:27:49.603932 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:49.603546    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7607-140230641700272" event={"ID":"0ac009f7-e845-437c-a247-19a858235f97","Type":"ContainerDied","Data":"0b513373fa641f4a0136ecdfe3195a972ca5d8853619c50aa7757d25039c3907"}
Jul 09 20:27:50.764895 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:50.764869    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7607-140230641700272"
Jul 09 20:27:50.835066 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:50.835027    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7607-140230641700272_0ac009f7-e845-437c-a247-19a858235f97/test-curl-7607-140230641700272/0.log"
Jul 09 20:27:50.865199 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:50.865098    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7607-140230641700272"]
Jul 09 20:27:50.867175 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:50.867154    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7607-140230641700272"]
Jul 09 20:27:50.873016 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:50.872991    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x9gz\" (UniqueName: \"kubernetes.io/projected/0ac009f7-e845-437c-a247-19a858235f97-kube-api-access-4x9gz\") pod \"0ac009f7-e845-437c-a247-19a858235f97\" (UID: \"0ac009f7-e845-437c-a247-19a858235f97\") "
Jul 09 20:27:50.875357 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:50.875332    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac009f7-e845-437c-a247-19a858235f97-kube-api-access-4x9gz" (OuterVolumeSpecName: "kube-api-access-4x9gz") pod "0ac009f7-e845-437c-a247-19a858235f97" (UID: "0ac009f7-e845-437c-a247-19a858235f97"). InnerVolumeSpecName "kube-api-access-4x9gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:27:50.973915 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:50.973875    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x9gz\" (UniqueName: \"kubernetes.io/projected/0ac009f7-e845-437c-a247-19a858235f97-kube-api-access-4x9gz\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:27:51.079069 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:51.079037    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac009f7-e845-437c-a247-19a858235f97" path="/var/lib/kubelet/pods/0ac009f7-e845-437c-a247-19a858235f97/volumes"
Jul 09 20:27:51.618617 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:51.618586    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7607-140230641700272"
Jul 09 20:27:51.618823 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:27:51.618589    2527 scope.go:117] "RemoveContainer" containerID="0b513373fa641f4a0136ecdfe3195a972ca5d8853619c50aa7757d25039c3907"
Jul 09 20:28:01.564907 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:01.564868    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"]
Jul 09 20:28:01.565486 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:01.565101    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m" podUID="e088a395-f669-4554-beff-af04fbbf639f" containerName="maas-api" containerID="cri-o://d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d" gracePeriod=30
Jul 09 20:28:02.014774 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.014748    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:28:02.082617 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.082576    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22t6k\" (UniqueName: \"kubernetes.io/projected/e088a395-f669-4554-beff-af04fbbf639f-kube-api-access-22t6k\") pod \"e088a395-f669-4554-beff-af04fbbf639f\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") "
Jul 09 20:28:02.082825 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.082675    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e088a395-f669-4554-beff-af04fbbf639f-maas-api-tls\") pod \"e088a395-f669-4554-beff-af04fbbf639f\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") "
Jul 09 20:28:02.082825 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.082760    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e088a395-f669-4554-beff-af04fbbf639f-openshift-service-ca\") pod \"e088a395-f669-4554-beff-af04fbbf639f\" (UID: \"e088a395-f669-4554-beff-af04fbbf639f\") "
Jul 09 20:28:02.083180 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.083147    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e088a395-f669-4554-beff-af04fbbf639f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e088a395-f669-4554-beff-af04fbbf639f" (UID: "e088a395-f669-4554-beff-af04fbbf639f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:28:02.084978 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.084952    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e088a395-f669-4554-beff-af04fbbf639f-kube-api-access-22t6k" (OuterVolumeSpecName: "kube-api-access-22t6k") pod "e088a395-f669-4554-beff-af04fbbf639f" (UID: "e088a395-f669-4554-beff-af04fbbf639f"). InnerVolumeSpecName "kube-api-access-22t6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:28:02.085163 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.085142    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e088a395-f669-4554-beff-af04fbbf639f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e088a395-f669-4554-beff-af04fbbf639f" (UID: "e088a395-f669-4554-beff-af04fbbf639f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:28:02.184767 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.184679    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e088a395-f669-4554-beff-af04fbbf639f-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:28:02.184767 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.184713    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e088a395-f669-4554-beff-af04fbbf639f-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:28:02.184767 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.184729    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22t6k\" (UniqueName: \"kubernetes.io/projected/e088a395-f669-4554-beff-af04fbbf639f-kube-api-access-22t6k\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:28:02.680021 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.679987    2527 generic.go:356] "Generic (PLEG): container finished" podID="e088a395-f669-4554-beff-af04fbbf639f" containerID="d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d" exitCode=0
Jul 09 20:28:02.680432 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.680036    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"
Jul 09 20:28:02.680432 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.680075    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m" event={"ID":"e088a395-f669-4554-beff-af04fbbf639f","Type":"ContainerDied","Data":"d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d"}
Jul 09 20:28:02.680432 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.680133    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m" event={"ID":"e088a395-f669-4554-beff-af04fbbf639f","Type":"ContainerDied","Data":"471cc4579b501cc7097a8bbe4d6b76ad14bd1f24926cdd73a1226686295b5053"}
Jul 09 20:28:02.680432 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.680163    2527 scope.go:117] "RemoveContainer" containerID="d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d"
Jul 09 20:28:02.690125 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.690104    2527 scope.go:117] "RemoveContainer" containerID="d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d"
Jul 09 20:28:02.690405 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:28:02.690386    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d\": container with ID starting with d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d not found: ID does not exist" containerID="d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d"
Jul 09 20:28:02.690456 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.690416    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d"} err="failed to get container status \"d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d\": rpc error: code = NotFound desc = could not find container \"d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d\": container with ID starting with d9c0dacb23a0ec7413554a0c686430e2b3198cd48d94154f7570b5dec654716d not found: ID does not exist"
Jul 09 20:28:02.701912 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.701886    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"]
Jul 09 20:28:02.706091 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:02.706071    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7b406b-788ff89f45-sjn9m"]
Jul 09 20:28:03.077437 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:03.077401    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e088a395-f669-4554-beff-af04fbbf639f" path="/var/lib/kubelet/pods/e088a395-f669-4554-beff-af04fbbf639f/volumes"
Jul 09 20:28:17.230031 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.229987    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"]
Jul 09 20:28:17.230621 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.230600    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e088a395-f669-4554-beff-af04fbbf639f" containerName="maas-api"
Jul 09 20:28:17.230740 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.230624    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="e088a395-f669-4554-beff-af04fbbf639f" containerName="maas-api"
Jul 09 20:28:17.230740 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.230673    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac009f7-e845-437c-a247-19a858235f97" containerName="test-curl-7607-140230641700272"
Jul 09 20:28:17.230740 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.230684    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac009f7-e845-437c-a247-19a858235f97" containerName="test-curl-7607-140230641700272"
Jul 09 20:28:17.230905 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.230837    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac009f7-e845-437c-a247-19a858235f97" containerName="test-curl-7607-140230641700272"
Jul 09 20:28:17.230905 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.230851    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="e088a395-f669-4554-beff-af04fbbf639f" containerName="maas-api"
Jul 09 20:28:17.235281 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.235257    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"
Jul 09 20:28:17.238316 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.238296    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5hwxb\""
Jul 09 20:28:17.244216 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.244193    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"]
Jul 09 20:28:17.341429 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.341394    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bcc3df6-c682-48f9-9ed5-35c3bded8e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c\" (UID: \"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"
Jul 09 20:28:17.442115 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.442078    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bcc3df6-c682-48f9-9ed5-35c3bded8e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c\" (UID: \"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"
Jul 09 20:28:17.442743 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.442724    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bcc3df6-c682-48f9-9ed5-35c3bded8e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c\" (UID: \"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"
Jul 09 20:28:17.547543 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.547505    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"
Jul 09 20:28:17.718402 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.718366    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"]
Jul 09 20:28:17.719715 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:28:17.719680    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bcc3df6_c682_48f9_9ed5_35c3bded8e0e.slice/crio-2d58f33bede233cefe22bd1134046147d6b0fcf38d253462f41f1f71afb37450 WatchSource:0}: Error finding container 2d58f33bede233cefe22bd1134046147d6b0fcf38d253462f41f1f71afb37450: Status 404 returned error can't find the container with id 2d58f33bede233cefe22bd1134046147d6b0fcf38d253462f41f1f71afb37450
Jul 09 20:28:17.767911 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:17.767866    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c" event={"ID":"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e","Type":"ContainerStarted","Data":"2d58f33bede233cefe22bd1134046147d6b0fcf38d253462f41f1f71afb37450"}
Jul 09 20:28:18.775515 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:18.775482    2527 generic.go:356] "Generic (PLEG): container finished" podID="3bcc3df6-c682-48f9-9ed5-35c3bded8e0e" containerID="18b761c1b17498a45ac2788351a4d06202a64ed257d72ed49e9206c42e173f3d" exitCode=0
Jul 09 20:28:18.775911 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:18.775568    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c" event={"ID":"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e","Type":"ContainerDied","Data":"18b761c1b17498a45ac2788351a4d06202a64ed257d72ed49e9206c42e173f3d"}
Jul 09 20:28:19.918383 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:19.918354    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"
Jul 09 20:28:20.066509 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:20.066422    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bcc3df6-c682-48f9-9ed5-35c3bded8e0e-maas-api-service-ca\") pod \"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e\" (UID: \"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e\") "
Jul 09 20:28:20.066882 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:20.066854    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bcc3df6-c682-48f9-9ed5-35c3bded8e0e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3bcc3df6-c682-48f9-9ed5-35c3bded8e0e" (UID: "3bcc3df6-c682-48f9-9ed5-35c3bded8e0e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:28:20.167479 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:20.167439    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bcc3df6-c682-48f9-9ed5-35c3bded8e0e-maas-api-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:28:20.789174 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:20.789144    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c"
Jul 09 20:28:20.789174 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:20.789177    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c" event={"ID":"3bcc3df6-c682-48f9-9ed5-35c3bded8e0e","Type":"ContainerDied","Data":"2d58f33bede233cefe22bd1134046147d6b0fcf38d253462f41f1f71afb37450"}
Jul 09 20:28:20.789410 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:20.789235    2527 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d58f33bede233cefe22bd1134046147d6b0fcf38d253462f41f1f71afb37450"
Jul 09 20:28:27.988236 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:27.988192    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"]
Jul 09 20:28:27.988753 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:27.988498    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx" podUID="83de1a9d-2b43-4f15-a11d-d8eb9b894054" containerName="maas-api" containerID="cri-o://37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0" gracePeriod=30
Jul 09 20:28:28.239409 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.239345    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:28:28.343479 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.343440    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83de1a9d-2b43-4f15-a11d-d8eb9b894054-openshift-service-ca\") pod \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") "
Jul 09 20:28:28.343479 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.343481    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls\") pod \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") "
Jul 09 20:28:28.343776 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.343616    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jjfd\" (UniqueName: \"kubernetes.io/projected/83de1a9d-2b43-4f15-a11d-d8eb9b894054-kube-api-access-5jjfd\") pod \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\" (UID: \"83de1a9d-2b43-4f15-a11d-d8eb9b894054\") "
Jul 09 20:28:28.343921 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.343897    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83de1a9d-2b43-4f15-a11d-d8eb9b894054-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83de1a9d-2b43-4f15-a11d-d8eb9b894054" (UID: "83de1a9d-2b43-4f15-a11d-d8eb9b894054"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 20:28:28.345819 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.345795    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83de1a9d-2b43-4f15-a11d-d8eb9b894054" (UID: "83de1a9d-2b43-4f15-a11d-d8eb9b894054"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 20:28:28.346003 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.345978    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83de1a9d-2b43-4f15-a11d-d8eb9b894054-kube-api-access-5jjfd" (OuterVolumeSpecName: "kube-api-access-5jjfd") pod "83de1a9d-2b43-4f15-a11d-d8eb9b894054" (UID: "83de1a9d-2b43-4f15-a11d-d8eb9b894054"). InnerVolumeSpecName "kube-api-access-5jjfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 20:28:28.444584 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.444547    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jjfd\" (UniqueName: \"kubernetes.io/projected/83de1a9d-2b43-4f15-a11d-d8eb9b894054-kube-api-access-5jjfd\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:28:28.444584 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.444581    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83de1a9d-2b43-4f15-a11d-d8eb9b894054-openshift-service-ca\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:28:28.444584 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.444595    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83de1a9d-2b43-4f15-a11d-d8eb9b894054-maas-api-tls\") on node \"ip-10-0-136-128.ec2.internal\" DevicePath \"\""
Jul 09 20:28:28.837656 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.837607    2527 generic.go:356] "Generic (PLEG): container finished" podID="83de1a9d-2b43-4f15-a11d-d8eb9b894054" containerID="37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0" exitCode=0
Jul 09 20:28:28.837845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.837676    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"
Jul 09 20:28:28.837845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.837697    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx" event={"ID":"83de1a9d-2b43-4f15-a11d-d8eb9b894054","Type":"ContainerDied","Data":"37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0"}
Jul 09 20:28:28.837845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.837756    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx" event={"ID":"83de1a9d-2b43-4f15-a11d-d8eb9b894054","Type":"ContainerDied","Data":"5a66ec85dc1c3b58d02f6db37962801066378a3d37ef93ef12c48f9797c63893"}
Jul 09 20:28:28.837845 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.837779    2527 scope.go:117] "RemoveContainer" containerID="37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0"
Jul 09 20:28:28.847659 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.847624    2527 scope.go:117] "RemoveContainer" containerID="37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0"
Jul 09 20:28:28.847918 ip-10-0-136-128 kubenswrapper[2527]: E0709 20:28:28.847901    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0\": container with ID starting with 37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0 not found: ID does not exist" containerID="37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0"
Jul 09 20:28:28.847970 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.847928    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0"} err="failed to get container status \"37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0\": rpc error: code = NotFound desc = could not find container \"37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0\": container with ID starting with 37be7cc123ac58edbbe82bdd95b9d18d51373bf4386f4660fe3a656214f2efb0 not found: ID does not exist"
Jul 09 20:28:28.860153 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.860124    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"]
Jul 09 20:28:28.864535 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:28.864510    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3452a7-856df5dc59-kl6dx"]
Jul 09 20:28:29.077776 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:29.077744    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83de1a9d-2b43-4f15-a11d-d8eb9b894054" path="/var/lib/kubelet/pods/83de1a9d-2b43-4f15-a11d-d8eb9b894054/volumes"
Jul 09 20:28:49.752331 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:49.752254    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-776898495d-wg5wh_e92cab95-3930-4bef-b08a-d219ee3b3bd8/copy-manifests/0.log"
Jul 09 20:28:49.767387 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:49.767361    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-776898495d-wg5wh_e92cab95-3930-4bef-b08a-d219ee3b3bd8/manager/0.log"
Jul 09 20:28:50.409000 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:50.408967    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29727135-9q2mb_d90dc504-0d11-4a54-aa2e-04dab08b6a51/cleanup/0.log"
Jul 09 20:28:50.647657 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:50.647612    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-8c04d3b2-bf5c5_db29c68b-385a-4b42-80dd-fe1886bf38e6/revoke-keys/0.log"
Jul 09 20:28:50.765884 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:50.765860    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-b7d55f-dh7k2_9247b046-27f5-4c11-b920-bc70a626c621/revoke-keys/0.log"
Jul 09 20:28:50.884563 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:50.884528    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-e062b5-z8nvm_feb1ca59-cce8-4c35-9882-70e720376908/revoke-keys/0.log"
Jul 09 20:28:51.005337 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:51.005305    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-e2f7b0b4-96f9t_0a5bf184-da9a-411f-acd1-9f5668e88995/revoke-keys/0.log"
Jul 09 20:28:51.246800 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:51.246720    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-54be63-dm9kj_66b2b952-8a16-4164-afde-296e3e712507/revoke-keys/0.log"
Jul 09 20:28:51.486996 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:51.486966    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-9bad9204-bvrnd_f38c9f55-b604-4d35-8fdd-0159f0297d49/revoke-keys/0.log"
Jul 09 20:28:51.737036 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:51.737001    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-3452a7-f5x7c_3bcc3df6-c682-48f9-9ed5-35c3bded8e0e/revoke-keys/0.log"
Jul 09 20:28:53.872657 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:53.872621    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-d4s7g_56875306-23fc-4064-8c01-8e5c96de29f2/manager/0.log"
Jul 09 20:28:54.778682 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:54.778600    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5f4bc74d-rx7kq_1dfb2417-24c2-47b0-bcdb-d4394d3356ca/kube-auth-proxy/0.log"
Jul 09 20:28:55.761682 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:55.761622    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-kdsfx_795a0d43-6157-4f80-a920-e660ed918d28/main/0.log"
Jul 09 20:28:55.885004 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:28:55.884977    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-l8k9h_0e701dd2-07eb-4bad-b5d1-26a1cac38028/main/0.log"
Jul 09 20:29:00.150430 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.150397    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l62zn/must-gather-lkg6z"]
Jul 09 20:29:00.150856 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.150776    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83de1a9d-2b43-4f15-a11d-d8eb9b894054" containerName="maas-api"
Jul 09 20:29:00.150856 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.150788    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="83de1a9d-2b43-4f15-a11d-d8eb9b894054" containerName="maas-api"
Jul 09 20:29:00.150856 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.150815    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bcc3df6-c682-48f9-9ed5-35c3bded8e0e" containerName="revoke-keys"
Jul 09 20:29:00.150856 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.150820    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bcc3df6-c682-48f9-9ed5-35c3bded8e0e" containerName="revoke-keys"
Jul 09 20:29:00.150989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.150891    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="83de1a9d-2b43-4f15-a11d-d8eb9b894054" containerName="maas-api"
Jul 09 20:29:00.150989 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.150900    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bcc3df6-c682-48f9-9ed5-35c3bded8e0e" containerName="revoke-keys"
Jul 09 20:29:00.154080 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.154064    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.156984 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.156962    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l62zn\"/\"openshift-service-ca.crt\""
Jul 09 20:29:00.157113 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.156963    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l62zn\"/\"kube-root-ca.crt\""
Jul 09 20:29:00.164050 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.163594    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-l62zn\"/\"default-dockercfg-tnls4\""
Jul 09 20:29:00.168245 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.168213    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l62zn/must-gather-lkg6z"]
Jul 09 20:29:00.256003 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.255964    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkkcb\" (UniqueName: \"kubernetes.io/projected/4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4-kube-api-access-wkkcb\") pod \"must-gather-lkg6z\" (UID: \"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4\") " pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.256182 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.256026    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4-must-gather-output\") pod \"must-gather-lkg6z\" (UID: \"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4\") " pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.357159 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.357123    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkkcb\" (UniqueName: \"kubernetes.io/projected/4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4-kube-api-access-wkkcb\") pod \"must-gather-lkg6z\" (UID: \"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4\") " pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.357355 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.357199    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4-must-gather-output\") pod \"must-gather-lkg6z\" (UID: \"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4\") " pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.357570 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.357550    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4-must-gather-output\") pod \"must-gather-lkg6z\" (UID: \"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4\") " pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.365624 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.365599    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkkcb\" (UniqueName: \"kubernetes.io/projected/4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4-kube-api-access-wkkcb\") pod \"must-gather-lkg6z\" (UID: \"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4\") " pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.471343 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.471256    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l62zn/must-gather-lkg6z"
Jul 09 20:29:00.607626 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:00.607600    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l62zn/must-gather-lkg6z"]
Jul 09 20:29:00.609685 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:29:00.609632    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bc668b5_31ce_46a3_b1dc_ca97ae4a38a4.slice/crio-8ac7d48be5e596d45e71242d1c994513958d8adbeecad2063c142b265fe32982 WatchSource:0}: Error finding container 8ac7d48be5e596d45e71242d1c994513958d8adbeecad2063c142b265fe32982: Status 404 returned error can't find the container with id 8ac7d48be5e596d45e71242d1c994513958d8adbeecad2063c142b265fe32982
Jul 09 20:29:01.009185 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:01.009147    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l62zn/must-gather-lkg6z" event={"ID":"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4","Type":"ContainerStarted","Data":"8ac7d48be5e596d45e71242d1c994513958d8adbeecad2063c142b265fe32982"}
Jul 09 20:29:02.019512 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:02.019459    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l62zn/must-gather-lkg6z" event={"ID":"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4","Type":"ContainerStarted","Data":"3c6e200a7398e43ad73dc9c3b152254881e05f5f92317346f0bdf305f1151a99"}
Jul 09 20:29:02.019512 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:02.019515    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l62zn/must-gather-lkg6z" event={"ID":"4bc668b5-31ce-46a3-b1dc-ca97ae4a38a4","Type":"ContainerStarted","Data":"7f8925d4c9708675f2205b8acd1ae83ccbe528f23ef08c7aa52df9669d6cd495"}
Jul 09 20:29:02.040285 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:02.040226    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l62zn/must-gather-lkg6z" podStartSLOduration=1.257373595 podStartE2EDuration="2.040205291s" totalImagesPullingTime="782.831696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 20:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:29:02.036880391 +0000 UTC m=+2953.438015202" watchObservedRunningTime="2026-07-09 20:29:02.040205291 +0000 UTC m=+2953.441340102"
Jul 09 20:29:03.012062 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:03.012033    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bjpjv_d0da2917-eb23-47fc-9db2-dd7737386e59/global-pull-secret-syncer/0.log"
Jul 09 20:29:03.135864 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:03.135825    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s4l69_87f73e8f-a2a4-4dfd-a685-ba51aa038002/konnectivity-agent/0.log"
Jul 09 20:29:03.186929 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:03.186882    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-128.ec2.internal_98ef2af5604f9c52421ea4f57e9babc4/haproxy/0.log"
Jul 09 20:29:09.068763 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:09.068727    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-d4s7g_56875306-23fc-4064-8c01-8e5c96de29f2/manager/0.log"
Jul 09 20:29:10.826201 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:10.826170    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3f3fc16d-d073-41d8-8f21-b07f370fea0c/alertmanager/0.log"
Jul 09 20:29:10.849915 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:10.849866    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3f3fc16d-d073-41d8-8f21-b07f370fea0c/config-reloader/0.log"
Jul 09 20:29:10.874304 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:10.874270    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3f3fc16d-d073-41d8-8f21-b07f370fea0c/kube-rbac-proxy-web/0.log"
Jul 09 20:29:10.903233 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:10.903206    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3f3fc16d-d073-41d8-8f21-b07f370fea0c/kube-rbac-proxy/0.log"
Jul 09 20:29:10.928092 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:10.928066    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3f3fc16d-d073-41d8-8f21-b07f370fea0c/kube-rbac-proxy-metric/0.log"
Jul 09 20:29:10.952602 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:10.952573    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3f3fc16d-d073-41d8-8f21-b07f370fea0c/prom-label-proxy/0.log"
Jul 09 20:29:10.975316 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:10.975267    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3f3fc16d-d073-41d8-8f21-b07f370fea0c/init-config-reloader/0.log"
Jul 09 20:29:11.051985 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.051957    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-f5t7k_a3fd7f1e-8d3e-4cf1-bc97-1975001adaad/kube-state-metrics/0.log"
Jul 09 20:29:11.070755 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.070725    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-f5t7k_a3fd7f1e-8d3e-4cf1-bc97-1975001adaad/kube-rbac-proxy-main/0.log"
Jul 09 20:29:11.100511 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.100431    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-f5t7k_a3fd7f1e-8d3e-4cf1-bc97-1975001adaad/kube-rbac-proxy-self/0.log"
Jul 09 20:29:11.134857 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.134829    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56889959ff-j7sh6_0c332ba1-106a-43d1-a7ad-77682b37f24c/metrics-server/0.log"
Jul 09 20:29:11.164360 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.164326    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-75sds_86a2f58d-7e0a-43e9-a8c6-2eb6105e0e39/monitoring-plugin/0.log"
Jul 09 20:29:11.280089 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.280028    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg4ml_1f31eedf-1c5a-4825-b852-3b66d081e2c1/node-exporter/0.log"
Jul 09 20:29:11.308112 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.308083    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg4ml_1f31eedf-1c5a-4825-b852-3b66d081e2c1/kube-rbac-proxy/0.log"
Jul 09 20:29:11.329554 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.329466    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg4ml_1f31eedf-1c5a-4825-b852-3b66d081e2c1/init-textfile/0.log"
Jul 09 20:29:11.430303 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.430231    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9657v_a05fdd56-7577-42c5-8345-1f4d1730331c/kube-rbac-proxy-main/0.log"
Jul 09 20:29:11.452538 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.452503    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9657v_a05fdd56-7577-42c5-8345-1f4d1730331c/kube-rbac-proxy-self/0.log"
Jul 09 20:29:11.474026 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.473996    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9657v_a05fdd56-7577-42c5-8345-1f4d1730331c/openshift-state-metrics/0.log"
Jul 09 20:29:11.508472 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.508437    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_596d64de-45da-47c0-af1f-e59dd020544a/prometheus/0.log"
Jul 09 20:29:11.536329 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.536300    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_596d64de-45da-47c0-af1f-e59dd020544a/config-reloader/0.log"
Jul 09 20:29:11.563601 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.563569    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_596d64de-45da-47c0-af1f-e59dd020544a/thanos-sidecar/0.log"
Jul 09 20:29:11.586331 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.586304    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_596d64de-45da-47c0-af1f-e59dd020544a/kube-rbac-proxy-web/0.log"
Jul 09 20:29:11.608846 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.608816    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_596d64de-45da-47c0-af1f-e59dd020544a/kube-rbac-proxy/0.log"
Jul 09 20:29:11.630781 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.630756    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_596d64de-45da-47c0-af1f-e59dd020544a/kube-rbac-proxy-thanos/0.log"
Jul 09 20:29:11.663618 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.663591    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_596d64de-45da-47c0-af1f-e59dd020544a/init-config-reloader/0.log"
Jul 09 20:29:11.708722 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.703211    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-5sxjl_2cc7e313-6b65-42d0-b0e1-13957d12977f/prometheus-operator/0.log"
Jul 09 20:29:11.728052 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.728027    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-5sxjl_2cc7e313-6b65-42d0-b0e1-13957d12977f/kube-rbac-proxy/0.log"
Jul 09 20:29:11.768388 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.768357    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-78z72_e74eff0b-696b-4449-89f2-efec0718a154/prometheus-operator-admission-webhook/0.log"
Jul 09 20:29:11.798743 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.798704    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64cb78f845-h46ld_a8658510-1390-4a81-8746-a441d5a0288e/telemeter-client/0.log"
Jul 09 20:29:11.824670 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.822260    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64cb78f845-h46ld_a8658510-1390-4a81-8746-a441d5a0288e/reload/0.log"
Jul 09 20:29:11.864745 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.864711    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64cb78f845-h46ld_a8658510-1390-4a81-8746-a441d5a0288e/kube-rbac-proxy/0.log"
Jul 09 20:29:11.895074 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.895020    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77cd7db9dc-w6wcj_8f863ade-b1d1-4b80-97f3-b1c54c607dcf/thanos-query/0.log"
Jul 09 20:29:11.915973 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.915938    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77cd7db9dc-w6wcj_8f863ade-b1d1-4b80-97f3-b1c54c607dcf/kube-rbac-proxy-web/0.log"
Jul 09 20:29:11.938637 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.938601    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77cd7db9dc-w6wcj_8f863ade-b1d1-4b80-97f3-b1c54c607dcf/kube-rbac-proxy/0.log"
Jul 09 20:29:11.962706 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.962621    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77cd7db9dc-w6wcj_8f863ade-b1d1-4b80-97f3-b1c54c607dcf/prom-label-proxy/0.log"
Jul 09 20:29:11.983855 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:11.983823    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77cd7db9dc-w6wcj_8f863ade-b1d1-4b80-97f3-b1c54c607dcf/kube-rbac-proxy-rules/0.log"
Jul 09 20:29:12.005888 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:12.005859    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-77cd7db9dc-w6wcj_8f863ade-b1d1-4b80-97f3-b1c54c607dcf/kube-rbac-proxy-metrics/0.log"
Jul 09 20:29:13.193724 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.193682    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"]
Jul 09 20:29:13.199739 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.199706    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.207391 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.207353    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"]
Jul 09 20:29:13.313559 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.313332    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-podres\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.313559 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.313405    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-sys\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.313559 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.313452    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccxc7\" (UniqueName: \"kubernetes.io/projected/8ec5a4da-df7b-480f-9841-5af8cf7932e5-kube-api-access-ccxc7\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.313559 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.313478    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-lib-modules\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.313559 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.313551    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-proc\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.414639 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.414594    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccxc7\" (UniqueName: \"kubernetes.io/projected/8ec5a4da-df7b-480f-9841-5af8cf7932e5-kube-api-access-ccxc7\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.414849 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.414668    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-lib-modules\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.414849 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.414711    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-proc\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.414849 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.414824    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-podres\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.415031 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.414868    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-sys\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.415031 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.414968    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-sys\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.415453 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.415426    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-lib-modules\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.415547 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.415496    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-proc\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.415613 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.415575    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ec5a4da-df7b-480f-9841-5af8cf7932e5-podres\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.427687 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.427627    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccxc7\" (UniqueName: \"kubernetes.io/projected/8ec5a4da-df7b-480f-9841-5af8cf7932e5-kube-api-access-ccxc7\") pod \"perf-node-gather-daemonset-n8p5c\" (UID: \"8ec5a4da-df7b-480f-9841-5af8cf7932e5\") " pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.519409 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.519372    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:13.727667 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:13.725229    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"]
Jul 09 20:29:13.732944 ip-10-0-136-128 kubenswrapper[2527]: W0709 20:29:13.732912    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8ec5a4da_df7b_480f_9841_5af8cf7932e5.slice/crio-3a2648701fa7e215396e73de20f915a24df4cd12667df039da59d43e1207a1ae WatchSource:0}: Error finding container 3a2648701fa7e215396e73de20f915a24df4cd12667df039da59d43e1207a1ae: Status 404 returned error can't find the container with id 3a2648701fa7e215396e73de20f915a24df4cd12667df039da59d43e1207a1ae
Jul 09 20:29:14.124463 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:14.124368    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c" event={"ID":"8ec5a4da-df7b-480f-9841-5af8cf7932e5","Type":"ContainerStarted","Data":"361136df3700d99ac879f2cd490bd93a8986ee1b015a356f200b8de5559874b9"}
Jul 09 20:29:14.124463 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:14.124442    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c" event={"ID":"8ec5a4da-df7b-480f-9841-5af8cf7932e5","Type":"ContainerStarted","Data":"3a2648701fa7e215396e73de20f915a24df4cd12667df039da59d43e1207a1ae"}
Jul 09 20:29:14.125254 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:14.125226    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:14.142188 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:14.142127    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c" podStartSLOduration=1.142097912 podStartE2EDuration="1.142097912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 20:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 20:29:14.14122494 +0000 UTC m=+2965.542359751" watchObservedRunningTime="2026-07-09 20:29:14.142097912 +0000 UTC m=+2965.543232723"
Jul 09 20:29:14.262214 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:14.262191    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78f856f565-t9j6z_07ef7226-8d24-42c9-b16a-55fdd36dbbb3/console/0.log"
Jul 09 20:29:15.893812 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:15.893784    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-868wr_84d6705f-f43f-411c-8f1c-90b05200da73/dns/0.log"
Jul 09 20:29:15.917007 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:15.916980    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-868wr_84d6705f-f43f-411c-8f1c-90b05200da73/kube-rbac-proxy/0.log"
Jul 09 20:29:16.091814 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:16.091785    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sd999_9dadcbf5-686f-4448-ab73-d05ec262206a/dns-node-resolver/0.log"
Jul 09 20:29:16.622318 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:16.622287    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-594f6499b-nz69n_ec51390a-ea19-4bda-a919-34fa8c5fbc8c/registry/0.log"
Jul 09 20:29:16.681796 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:16.681766    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nfv4x_ea4bda2e-4a85-4410-88a0-843cae158512/node-ca/0.log"
Jul 09 20:29:17.899502 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:17.899468    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5f4bc74d-rx7kq_1dfb2417-24c2-47b0-bcdb-d4394d3356ca/kube-auth-proxy/0.log"
Jul 09 20:29:18.816915 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:18.816880    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqt7l_c8e39e24-9e22-41ef-ba72-95c1c7490e92/serve-healthcheck-canary/0.log"
Jul 09 20:29:19.412017 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:19.411980    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmfjg_4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc/kube-rbac-proxy/0.log"
Jul 09 20:29:19.433780 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:19.433753    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmfjg_4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc/exporter/0.log"
Jul 09 20:29:19.457797 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:19.457763    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmfjg_4fa4397f-ec7d-4c32-aab1-f79e9e4d3dbc/extractor/0.log"
Jul 09 20:29:21.151307 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:21.151274    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-l62zn/perf-node-gather-daemonset-n8p5c"
Jul 09 20:29:22.310098 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:22.310057    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-776898495d-wg5wh_e92cab95-3930-4bef-b08a-d219ee3b3bd8/manager/0.log"
Jul 09 20:29:22.327536 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:22.327503    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-776898495d-wg5wh_e92cab95-3930-4bef-b08a-d219ee3b3bd8/copy-manifests/0.log"
Jul 09 20:29:23.675141 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:23.675112    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-748698b87b-8qr9s_49efb049-1b9e-4ff8-a111-9cbe7d9e2033/manager/0.log"
Jul 09 20:29:24.034573 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:24.034543    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5f4bc74d-rx7kq_1dfb2417-24c2-47b0-bcdb-d4394d3356ca/kube-auth-proxy/0.log"
Jul 09 20:29:25.541232 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:25.541180    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d5f4bc74d-rx7kq_1dfb2417-24c2-47b0-bcdb-d4394d3356ca/kube-auth-proxy/0.log"
Jul 09 20:29:30.084501 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.084464    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6t2f8_b76d8e50-2d66-4da1-b546-db2837e4aaf9/kube-multus/0.log"
Jul 09 20:29:30.277479 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.277450    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9c2zf_b7a72406-3b3d-4535-b5ba-8376627edc03/kube-multus-additional-cni-plugins/0.log"
Jul 09 20:29:30.299462 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.299430    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9c2zf_b7a72406-3b3d-4535-b5ba-8376627edc03/egress-router-binary-copy/0.log"
Jul 09 20:29:30.320909 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.320884    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9c2zf_b7a72406-3b3d-4535-b5ba-8376627edc03/cni-plugins/0.log"
Jul 09 20:29:30.342784 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.342716    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9c2zf_b7a72406-3b3d-4535-b5ba-8376627edc03/bond-cni-plugin/0.log"
Jul 09 20:29:30.365422 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.365394    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9c2zf_b7a72406-3b3d-4535-b5ba-8376627edc03/routeoverride-cni/0.log"
Jul 09 20:29:30.386641 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.386611    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9c2zf_b7a72406-3b3d-4535-b5ba-8376627edc03/whereabouts-cni-bincopy/0.log"
Jul 09 20:29:30.407703 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.407672    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9c2zf_b7a72406-3b3d-4535-b5ba-8376627edc03/whereabouts-cni/0.log"
Jul 09 20:29:30.657026 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.656948    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-42kwj_31b31734-c0f5-4653-9893-c3bb20fecb2c/network-metrics-daemon/0.log"
Jul 09 20:29:30.676041 ip-10-0-136-128 kubenswrapper[2527]: I0709 20:29:30.676015    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-42kwj_31b31734-c0f5-4653-9893-c3bb20fecb2c/kube-rbac-proxy/0.log"