Jul 15 19:56:33.014635 ip-10-0-130-52 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 15 19:56:33.014647 ip-10-0-130-52 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 15 19:56:33.014656 ip-10-0-130-52 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 15 19:56:33.014960 ip-10-0-130-52 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 15 19:56:43.169699 ip-10-0-130-52 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 15 19:56:43.169718 ip-10-0-130-52 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c43960607bd24740ac1638c4dc07d632 --
Jul 15 19:58:36.998663 ip-10-0-130-52 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 19:58:38.841044 ip-10-0-130-52 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:58:38.841044 ip-10-0-130-52 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 19:58:38.841044 ip-10-0-130-52 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:58:38.841044 ip-10-0-130-52 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 19:58:38.841044 ip-10-0-130-52 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:58:38.844231 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.844133    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 19:58:38.847641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847626    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:58:38.847641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847641    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847645    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847648    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847651    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847654    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847657    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847660    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847663    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847665    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847669    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847672    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847674    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847681    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847684    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847687    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847689    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847692    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847695    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847698    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847700    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:58:38.847717 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847703    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847705    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847708    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847711    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847714    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847716    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847721    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847726    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847730    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847733    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847736    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847739    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847742    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847745    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847747    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847751    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847753    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847756    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847759    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:58:38.848192 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847761    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847764    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847766    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847769    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847779    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847782    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847784    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847787    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847790    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847794    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847796    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847799    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847801    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847804    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847806    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847809    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847811    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847814    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847816    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847819    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:58:38.848639 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847822    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847825    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847827    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847830    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847833    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847835    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847838    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847841    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847843    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847846    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847848    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847851    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847853    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847856    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847859    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847861    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847864    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847866    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847869    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847871    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:58:38.849159 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847874    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847876    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847880    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847882    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847885    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847888    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847891    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847893    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847895    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847898    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847900    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847905    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847907    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847910    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847912    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847915    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847918    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847922    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.847926    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:58:38.849630 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848651    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848658    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848662    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848665    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848668    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848671    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848674    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848677    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848679    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848682    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848685    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848688    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848691    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848694    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848696    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848699    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848701    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848704    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848706    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848709    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:58:38.850094 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848711    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848714    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848717    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848720    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848723    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848726    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848728    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848731    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848735    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848738    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848741    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848744    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848747    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848749    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848752    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848755    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848758    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848760    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848763    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:58:38.850575 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848765    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848768    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848770    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848773    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848776    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848782    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848785    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848788    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848790    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848793    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848801    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848804    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848807    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848809    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848812    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848815    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848818    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848820    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848823    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848826    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:58:38.851058 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848829    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848832    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848834    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848837    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848840    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848842    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848845    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848847    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848850    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848852    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848855    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848857    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848860    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848862    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848865    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848867    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848870    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848872    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848875    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848878    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:58:38.851542 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848882    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848886    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848888    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848891    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848894    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848896    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848899    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848902    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848904    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848907    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848910    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848912    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848915    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848917    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848920    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848923    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848927    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848930    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848933    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:58:38.852022 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.848935    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849305    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849311    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849314    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849317    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849320    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849322    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849325    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849327    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849330    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849333    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849336    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849339    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849341    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849344    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849347    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849350    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849352    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849355    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849357    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:58:38.852479 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849360    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849363    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849365    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849368    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849370    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849373    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849376    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849378    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849380    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849383    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849385    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849388    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849390    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849393    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849395    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849398    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849400    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849403    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849405    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849407    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:58:38.852983 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849410    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849412    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849414    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849417    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849420    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849422    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849425    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849428    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849431    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849433    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849435    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849438    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849441    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849443    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849446    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849448    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849451    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849454    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849456    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849460    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:58:38.853451 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849464    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849467    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849470    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849472    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849475    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849478    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849480    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849483    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849485    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849488    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849491    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849494    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849497    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849499    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849501    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849504    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849507    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849510    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849513    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:58:38.853925 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849515    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849518    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849520    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849523    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849527    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849530    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849533    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849535    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849539    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849542    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849545    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849547    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849550    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849552    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849555    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849557    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849560    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849563    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849565    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:58:38.854420 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849568    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.849570    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849639    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849646    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849652    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849657    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849663    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849666    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849670    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849674    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849677    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849680    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849686    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849690    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849693    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849696    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849699    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849702    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849704    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849707    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849712    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849715    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849718    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849721    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 19:58:38.854868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849724    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849728    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849731    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849734    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849737    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849740    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849742    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849745    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849748    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849751    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849755    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849758    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849760    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849763    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849767    2562 flags.go:64] FLAG: --enable-server="true"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849770    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849774    2562 flags.go:64] FLAG: --event-burst="100"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849777    2562 flags.go:64] FLAG: --event-qps="50"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849780    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849783    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849787    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849792    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849795    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849798    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849800    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 15 19:58:38.855475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849803    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849806    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849809    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849812    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849815    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849818    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849820    2562 flags.go:64] FLAG: --feature-gates=""
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849824    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849827    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849830    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849833    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849836    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849839    2562 flags.go:64] FLAG: --help="false"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849842    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-130-52.ec2.internal"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849845    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849848    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849851    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849854    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849857    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849860    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849862    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849866    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849869    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 19:58:38.856108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849872    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849875    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849878    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849880    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849885    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849888    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849891    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849894    2562 flags.go:64] FLAG: --lock-file=""
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849897    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849900    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849903    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849908    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849910    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849913    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849916    2562 flags.go:64] FLAG: --logging-format="text"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849919    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849922    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849925    2562 flags.go:64] FLAG: --manifest-url=""
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849928    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849932    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849935    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849939    2562 flags.go:64] FLAG: --max-pods="110"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849941    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849944    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849947    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 19:58:38.856752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849950    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849953    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849956    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849976    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849986    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849991    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849994    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.849997    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850000    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850005    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850008    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850012    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850015    2562 flags.go:64] FLAG: --port="10250"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850018    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850022    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cf3b7c85b0665d24"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850025    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850028    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850031    2562 flags.go:64] FLAG: --register-node="true"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850034    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850038    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850040    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850043    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850046    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850049    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850052    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 19:58:38.857659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850055    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850058    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850061    2562 flags.go:64] FLAG: --runonce="false"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850063    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850066    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850069    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850072    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850075    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850078    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850081    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850083    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850086    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850089    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850093    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850096    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850099    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850102    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850105    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850112    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850115    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850118    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850122    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850126    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850129    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850132    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 19:58:38.858295 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850135    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850137    2562 flags.go:64] FLAG: --v="2"
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850141    2562 flags.go:64] FLAG: --version="false"
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850145    2562 flags.go:64] FLAG: --vmodule=""
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850149    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850152    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850265    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850269    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850272    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850275    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850278    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850281    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850283    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850286    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850289    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850291    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850294    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850296    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850299    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850302    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850306    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:58:38.858938 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850308    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850311    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850314    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850316    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850320    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850322    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850325    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850328    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850330    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850333    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850336    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850339    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850343    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850346    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850348    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850351    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850353    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850355    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850358    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:58:38.859485 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850361    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850363    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850365    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850368    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850370    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850373    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850376    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850379    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850381    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850384    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850388    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850392    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850396    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850399    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850402    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850405    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850408    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850412    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850415    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850418    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:58:38.859941 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850420    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850423    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850426    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850429    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850431    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850434    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850436    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850439    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850441    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850444    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850446    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850449    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850451    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850455    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850458    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850461    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850464    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850467    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850469    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850472    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:58:38.860430 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850474    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850477    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850479    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850482    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850486    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850488    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850491    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850493    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850496    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850500    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850503    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850506    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850508    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850511    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850513    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850516    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850519    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850521    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850524    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850526    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:58:38.860911 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850529    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850531    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850534    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850536    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.850538    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.850544    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.858346    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.858361    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.858987    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.858998    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859057    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859062    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859067    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859072    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:58:38.861486 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859075    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859078    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859082    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859085    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859088    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859091    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859094    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859096    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859099    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859102    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859104    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859107    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859109    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859112    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859114    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859117    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859119    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859122    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859125    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:58:38.861831 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859133    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859136    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859139    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859142    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859144    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859147    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859150    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859154    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859158    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859162    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859164    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859167    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859170    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859172    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859175    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859177    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859181    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859184    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859187    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:58:38.862292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859191    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859194    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859197    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859200    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859202    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859205    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859208    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859217    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859219    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859222    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859224    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859227    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859229    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859232    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859234    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859237    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859239    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859242    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859245    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859248    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:58:38.862804 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859251    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859253    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859256    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859259    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859262    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859265    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859268    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859270    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859273    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859275    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859278    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859280    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859283    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859286    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859288    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859291    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859293    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859297    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859299    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859302    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:58:38.863292 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859304    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859307    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859309    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859312    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859315    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859317    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859320    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859322    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859325    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859327    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859330    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859333    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859335    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859338    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859341    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859343    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:58:38.863805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859346    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.859351    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859479    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859485    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859488    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859493    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859497    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859501    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859504    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859506    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859510    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859512    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859515    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859517    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859520    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:58:38.864274 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859523    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859526    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859528    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859531    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859534    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859536    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859539    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859542    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859544    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859547    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859550    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859554    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859558    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859561    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859564    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859566    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859569    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859572    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859574    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:58:38.864641 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859577    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859579    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859582    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859584    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859587    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859589    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859592    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859594    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859597    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859599    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859602    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859604    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859607    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859610    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859613    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859615    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859618    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859621    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859624    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859627    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:58:38.865194 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859629    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859632    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859634    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859637    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859640    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859642    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859645    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859647    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859650    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859652    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859655    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859657    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859660    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859662    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859664    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859667    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859669    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859672    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859675    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859677    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:58:38.865764 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859679    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859682    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859684    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859687    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859689    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859692    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859695    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859697    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859700    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859702    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859705    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859707    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859710    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859713    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859715    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859718    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859720    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859723    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859727    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859730    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:58:38.866568 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859733    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859736    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859739    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859741    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859744    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859747    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:38.859749    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.859754    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.859894    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.862494    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.863667    2562 server.go:1019] "Starting client certificate rotation"
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.863780    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.863831    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:58:38.867096 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.866679    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:58:38.867866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.867845    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:58:38.881409 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.881384    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 15 19:58:38.884108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.884096    2562 log.go:25] "Validated CRI v1 image API"
Jul 15 19:58:38.884989 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.884956    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 19:58:38.888042 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.888023    2562 fs.go:135] Filesystem UUIDs: map[505f2634-cf76-4d14-b168-6e01a6700a4b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f0a0fa50-6d2b-4caf-b09e-3c1820ddf614:/dev/nvme0n1p4]
Jul 15 19:58:38.888112 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.888040    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 19:58:38.890697 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.890680    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:58:38.894461 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.894345    2562 manager.go:216] Machine: {Timestamp:2026-07-15 19:58:38.892310839 +0000 UTC m=+0.340683929 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3085889 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec222cfedbe682ca28d28f709f50116d SystemUUID:ec222cfe-dbe6-82ca-28d2-8f709f50116d BootID:c4396060-7bd2-4740-ac16-38c4dc07d632 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:28:27:6c:ce:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:28:27:6c:ce:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:8d:e5:9b:8c:85 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 19:58:38.894461 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.894454    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 19:58:38.894574 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.894527    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 19:58:38.898566 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.898369    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 19:58:38.898713 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.898570    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-52.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 19:58:38.899863 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.899851    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 19:58:38.899905 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.899866    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 19:58:38.899905 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.899873    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:58:38.901717 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.901707    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:58:38.902463 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.902453    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 19:58:38.905861 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.905850    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:58:38.906009 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.905999    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 19:58:38.906045 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.906032    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 19:58:38.906073 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.906045    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 19:58:38.906073 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.906069    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 19:58:38.906122 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.906082    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 15 19:58:38.906122 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.906094    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 19:58:38.906519 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.906506    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:58:38.906566 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.906529    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:58:38.908434 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.908414    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 19:58:38.909773 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.909761    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 19:58:38.909816 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.909791    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 19:58:38.910359 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910349    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 19:58:38.910404 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910364    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 19:58:38.910404 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910371    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 19:58:38.910404 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910376    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 19:58:38.910404 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910384    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 19:58:38.910404 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910390    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 19:58:38.910533 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910518    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 19:58:38.910533 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910525    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 19:58:38.910533 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910532    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 19:58:38.910607 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910539    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 19:58:38.910607 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910548    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 19:58:38.910607 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.910556    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 19:58:38.912160 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.912151    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 19:58:38.912198 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.912169    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 19:58:38.915267 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.915256    2562 server.go:1268] "Started kubelet"
Jul 15 19:58:38.915774 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.915553    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 19:58:38.915774 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.915553    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 19:58:38.915774 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.915750    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 19:58:38.916241 ip-10-0-130-52 systemd[1]: Started Kubernetes Kubelet.
Jul 15 19:58:38.917284 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.917256    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 19:58:38.917510 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.917488    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 19:58:38.919677 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.919633    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 19:58:38.919677 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.919663    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-52.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 19:58:38.919813 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.919713    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 19:58:38.919938 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.919920    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 19:58:38.921160 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.921140    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 19:58:38.921436 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.921420    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4nnc8"
Jul 15 19:58:38.921783 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.921766    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 19:58:38.921877 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.921855    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:38.922510 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922495    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 19:58:38.922510 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922497    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 19:58:38.922662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922519    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 19:58:38.922662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922595    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 19:58:38.922662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922606    2562 factory.go:55] Registering systemd factory
Jul 15 19:58:38.922662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922615    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 15 19:58:38.922662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922620    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 19:58:38.922662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922627    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 19:58:38.922900 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922826    2562 factory.go:153] Registering CRI-O factory
Jul 15 19:58:38.922900 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922842    2562 factory.go:223] Registration of the crio container factory successfully
Jul 15 19:58:38.922900 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922868    2562 factory.go:103] Registering Raw factory
Jul 15 19:58:38.922900 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.922877    2562 manager.go:1196] Started watching for new ooms in manager
Jul 15 19:58:38.923216 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.923204    2562 manager.go:349] Starting recovery of all containers
Jul 15 19:58:38.930318 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.930275    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4nnc8"
Jul 15 19:58:38.934432 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.934314    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 19:58:38.935053 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.935014    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 19:58:38.935473 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.934391    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-52.ec2.internal.18c28ee58a8c130b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-52.ec2.internal,UID:ip-10-0-130-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-52.ec2.internal,},FirstTimestamp:2026-07-15 19:58:38.915228427 +0000 UTC m=+0.363601514,LastTimestamp:2026-07-15 19:58:38.915228427 +0000 UTC m=+0.363601514,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-52.ec2.internal,}"
Jul 15 19:58:38.937119 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.937106    2562 manager.go:354] Recovery completed
Jul 15 19:58:38.941317 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.941306    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:58:38.945107 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.945087    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:58:38.945169 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.945124    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:58:38.945169 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.945141    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:58:38.945776 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.945761    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 19:58:38.945776 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.945773    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 19:58:38.945932 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.945787    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:58:38.945932 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.945802    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:58:38.946502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.946482    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:58:38.946550 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.946508    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:58:38.946550 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.946519    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:58:38.946847 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.946836    2562 policy_none.go:49] "None policy: Start"
Jul 15 19:58:38.946907 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.946851    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 19:58:38.946907 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.946861    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 19:58:38.947803 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.947794    2562 policy_none.go:47] "Start"
Jul 15 19:58:38.980627 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.980466    2562 manager.go:337] "Starting Device Plugin manager"
Jul 15 19:58:38.980710 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.980659    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 19:58:38.980710 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.980672    2562 server.go:87] "Starting device plugin registration server"
Jul 15 19:58:38.980920 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.980909    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 19:58:38.980950 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.980923    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 19:58:38.981021 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.980996    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 19:58:38.981095 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.981081    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 19:58:38.981140 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:38.981097    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 19:58:38.981673 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.981655    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 19:58:38.981761 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:38.981703    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.071218 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.071143    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 19:58:39.072293 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.072279    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 19:58:39.072358 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.072316    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 19:58:39.072358 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.072340    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 19:58:39.072459 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.072372    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 19:58:39.074625 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.074607    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:58:39.081327 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.081314    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:58:39.083333 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.083319    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:58:39.083421 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.083351    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:58:39.083421 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.083368    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:58:39.083421 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.083396    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.089549 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.089534    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.089610 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.089555    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-52.ec2.internal\": node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.110106 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.110085    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.172570 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.172546    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"]
Jul 15 19:58:39.172707 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.172634    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:58:39.174791 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.174774    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:58:39.174881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.174808    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:58:39.174881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.174822    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:58:39.176119 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.176104    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:58:39.176230 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.176210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.176310 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.176248    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:58:39.176963 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.176949    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:58:39.177061 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.176991    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:58:39.177061 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.177005    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:58:39.177153 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.177060    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:58:39.177153 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.177087    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:58:39.177153 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.177099    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:58:39.178247 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.178232    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.178324 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.178261    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:58:39.179107 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.179093    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:58:39.179170 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.179116    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:58:39.179170 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.179125    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:58:39.203732 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.203710    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-52.ec2.internal\" not found" node="ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.207990 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.207959    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-52.ec2.internal\" not found" node="ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.210323 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.210311    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.224724 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.224704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3d8921e521c473a7998a09a95a39b65-config\") pod \"kube-apiserver-proxy-ip-10-0-130-52.ec2.internal\" (UID: \"a3d8921e521c473a7998a09a95a39b65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.224820 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.224730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f454b558f2d5d65067cce20f5bd5b71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7f454b558f2d5d65067cce20f5bd5b71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.224820 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.224753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f454b558f2d5d65067cce20f5bd5b71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7f454b558f2d5d65067cce20f5bd5b71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.311064 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.311037    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.325083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.325021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f454b558f2d5d65067cce20f5bd5b71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7f454b558f2d5d65067cce20f5bd5b71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.325083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.325055    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f454b558f2d5d65067cce20f5bd5b71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7f454b558f2d5d65067cce20f5bd5b71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.325083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.325079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3d8921e521c473a7998a09a95a39b65-config\") pod \"kube-apiserver-proxy-ip-10-0-130-52.ec2.internal\" (UID: \"a3d8921e521c473a7998a09a95a39b65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.325247 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.325115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a3d8921e521c473a7998a09a95a39b65-config\") pod \"kube-apiserver-proxy-ip-10-0-130-52.ec2.internal\" (UID: \"a3d8921e521c473a7998a09a95a39b65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.325247 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.325124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f454b558f2d5d65067cce20f5bd5b71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7f454b558f2d5d65067cce20f5bd5b71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.325247 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.325115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f454b558f2d5d65067cce20f5bd5b71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal\" (UID: \"7f454b558f2d5d65067cce20f5bd5b71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.411566 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.411528    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.507068 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.507026    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.511691 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.511323    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:39.512421 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.512405    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.613049 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.612935    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.713513 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.713481    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.814049 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.814019    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.863545 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.863478    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 19:58:39.863927 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.863609    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 19:58:39.914979 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:39.914936    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:39.935428 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.935386    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:53:38 +0000 UTC" deadline="2028-03-18 20:44:30.741243315 +0000 UTC"
Jul 15 19:58:39.935428 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.935422    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14688h45m50.80582495s"
Jul 15 19:58:39.945376 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:39.945356    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:58:40.016056 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:40.016010    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-52.ec2.internal\" not found"
Jul 15 19:58:40.044634 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:40.044606    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d8921e521c473a7998a09a95a39b65.slice/crio-503fa928a04adf964e902faf66f2fc4afffab81cd71e3c29b22c115435a2dacc WatchSource:0}: Error finding container 503fa928a04adf964e902faf66f2fc4afffab81cd71e3c29b22c115435a2dacc: Status 404 returned error can't find the container with id 503fa928a04adf964e902faf66f2fc4afffab81cd71e3c29b22c115435a2dacc
Jul 15 19:58:40.044957 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:40.044935    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f454b558f2d5d65067cce20f5bd5b71.slice/crio-01f07bf21b504e16bf74cda53aef4cf04aa6b24b80e7ab9b3788624786d9cdc2 WatchSource:0}: Error finding container 01f07bf21b504e16bf74cda53aef4cf04aa6b24b80e7ab9b3788624786d9cdc2: Status 404 returned error can't find the container with id 01f07bf21b504e16bf74cda53aef4cf04aa6b24b80e7ab9b3788624786d9cdc2
Jul 15 19:58:40.048445 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.048430    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:58:40.075439 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.075396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" event={"ID":"a3d8921e521c473a7998a09a95a39b65","Type":"ContainerStarted","Data":"503fa928a04adf964e902faf66f2fc4afffab81cd71e3c29b22c115435a2dacc"}
Jul 15 19:58:40.076368 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.076348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" event={"ID":"7f454b558f2d5d65067cce20f5bd5b71","Type":"ContainerStarted","Data":"01f07bf21b504e16bf74cda53aef4cf04aa6b24b80e7ab9b3788624786d9cdc2"}
Jul 15 19:58:40.108209 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.108191    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:58:40.123008 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.122951    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:40.135936 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.135919    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:58:40.136769 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.136757    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal"
Jul 15 19:58:40.146174 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.146162    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:58:40.260246 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.260222    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:58:40.907299 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.907270    2562 apiserver.go:52] "Watching apiserver"
Jul 15 19:58:40.920705 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.920679    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 19:58:40.921062 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.921036    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-j8dpm","kube-system/konnectivity-agent-bbmcl","openshift-multus/multus-b4zjt","openshift-multus/network-metrics-daemon-694gf","openshift-network-diagnostics/network-check-target-ft87c","openshift-network-operator/iptables-alerter-ksr9b","openshift-ovn-kubernetes/ovnkube-node-vr9sl","kube-system/global-pull-secret-syncer-zh2mc","kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f","openshift-cluster-node-tuning-operator/tuned-lvn92","openshift-dns/node-resolver-lwq7z","openshift-image-registry/node-ca-8z8br","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal"]
Jul 15 19:58:40.921707 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.921692    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 19:58:40.922680 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.922664    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.923948 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.923922    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:40.925327 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.925309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:40.926670 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.926640    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:40.926767 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:40.926744    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:40.927947 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.927928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:40.928048 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:40.928009    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:40.929252 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.929233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:40.930275 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 19:58:40.930350 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930210    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 19:58:40.930406 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930211    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 19:58:40.930406 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930286    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 19:58:40.930406 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930368    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 19:58:40.930559 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 19:58:40.930634 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930618    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 19:58:40.930864 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.930850    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:40.930944 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:40.930906    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:40.932072 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.932056    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:40.932520 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.932501    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 19:58:40.933362 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.933148    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 19:58:40.933362 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.933258    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c6d6v\""
Jul 15 19:58:40.933362 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.933272    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 19:58:40.933362 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.933275    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 19:58:40.933593 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.933469    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:40.934180 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934045    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 19:58:40.934180 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934072    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-log-socket\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934180 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-systemd\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934180 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-etc-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934190    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-node-log\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934258    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27fda661-b8a4-4bed-b977-ccf99ecc406e-serviceca\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-systemd-units\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovnkube-script-lib\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934366    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m6wcr\""
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnfsc\" (UniqueName: \"kubernetes.io/projected/27fda661-b8a4-4bed-b977-ccf99ecc406e-kube-api-access-mnfsc\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:40.934411 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-kubelet\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-slash\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-var-lib-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934492    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-ovn\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-cni-bin\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-cni-netd\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovnkube-config\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-run-netns\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-env-overrides\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovn-node-metrics-cert\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdfxs\" (UniqueName: \"kubernetes.io/projected/1228f9c2-e269-4f64-a677-fae2b38ed3d7-kube-api-access-hdfxs\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:40.934881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27fda661-b8a4-4bed-b977-ccf99ecc406e-host\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:40.935357 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.934875    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:40.938051 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.936653    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hq6xn\""
Jul 15 19:58:40.938051 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.936867    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 19:58:40.938051 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.937276    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:58:40.938051 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.937721    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 19:58:40.938051 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.937875    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9lbbx\""
Jul 15 19:58:40.939312 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.939294    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:40.940978 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.940948    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:40.945550 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.945282    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 19:58:40.946141 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946121    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n7csg\""
Jul 15 19:58:40.946239 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946176    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 19:58:40.946239 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946228    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 19:58:40.946423 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946383    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hmmcg\""
Jul 15 19:58:40.946499 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946431    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 19:58:40.946499 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946466    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 19:58:40.946630 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946574    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 19:58:40.946700 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946624    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:58:40.946700 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946639    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 19:58:40.946700 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.946645    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 19:58:40.947730 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.947566    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7tmjg\""
Jul 15 19:58:40.947730 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.947605    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zj8ql\""
Jul 15 19:58:40.947730 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.947626    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 19:58:40.947730 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.947630    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 19:58:40.947730 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.947659    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 19:58:40.947730 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.947612    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rj857\""
Jul 15 19:58:40.964302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.964282    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:58:40.973553 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:40.973538    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:58:41.023598 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.023575    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 19:58:41.034856 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.034833    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bqv9f"
Jul 15 19:58:41.034984 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.034868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.034984 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.034902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-cnibin\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.034984 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.034927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-device-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.034984 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.034950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.034987    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.034994    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-slash\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-slash\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035034    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-kubelet\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f6563d9-f278-4f12-a251-c7808bbe1f85-konnectivity-ca\") pod \"konnectivity-agent-bbmcl\" (UID: \"6f6563d9-f278-4f12-a251-c7808bbe1f85\") " pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-systemd\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035159    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035188    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-systemd\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035199 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-system-cni-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-netns\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035270    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-systemd\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-daemon-config\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-sys\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-kubelet\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-ovn\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-kubelet\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-cni-bin\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-cni-bin\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-ovn\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-cni-netd\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-cni-netd\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-cni-bin\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-hostroot\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysctl-d\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.035656 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-var-lib-kubelet\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035653    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-k8s-cni-cncf-io\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-etc-kubernetes\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n2hq\" (UniqueName: \"kubernetes.io/projected/31004923-49f0-49f5-9c4e-41abeffb6da6-kube-api-access-9n2hq\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035763    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-tuned\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-tmp-dir\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035810    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jfcr\" (UniqueName: \"kubernetes.io/projected/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-kube-api-access-9jfcr\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035856    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-log-socket\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-node-log\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035921    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-log-socket\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-cni-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.035983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-conf-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-node-log\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036037    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-run\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-host\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.036437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovnkube-script-lib\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41d3dc94-0044-4326-92be-d1550a0502b1-kubelet-config\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-os-release\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-run-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f6563d9-f278-4f12-a251-c7808bbe1f85-agent-certs\") pod \"konnectivity-agent-bbmcl\" (UID: \"6f6563d9-f278-4f12-a251-c7808bbe1f85\") " pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036242    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-var-lib-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31004923-49f0-49f5-9c4e-41abeffb6da6-cni-binary-copy\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036306    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-var-lib-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtq5m\" (UniqueName: \"kubernetes.io/projected/6e1a0211-5b8b-4bf4-ae26-362b79b02546-kube-api-access-rtq5m\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98b238a4-272b-43c3-bcb0-b5cead6a44c0-iptables-alerter-script\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-kubernetes\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036428    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-run-netns\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovn-node-metrics-cert\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-run-netns\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27fda661-b8a4-4bed-b977-ccf99ecc406e-host\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:41.037243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27fda661-b8a4-4bed-b977-ccf99ecc406e-host\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-socket-dir-parent\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98b238a4-272b-43c3-bcb0-b5cead6a44c0-host-slash\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-modprobe-d\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-etc-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27fda661-b8a4-4bed-b977-ccf99ecc406e-serviceca\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovnkube-script-lib\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036777    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-etc-openvswitch\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-sys-fs\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036858    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pczf7\" (UniqueName: \"kubernetes.io/projected/24c2d436-4147-4c2d-b062-77c339b3093c-kube-api-access-pczf7\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5rxv\" (UniqueName: \"kubernetes.io/projected/d285714b-fe13-43e7-b7e3-1e7011dd666b-kube-api-access-f5rxv\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-systemd-units\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnfsc\" (UniqueName: \"kubernetes.io/projected/27fda661-b8a4-4bed-b977-ccf99ecc406e-kube-api-access-mnfsc\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036946    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-system-cni-dir\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-socket-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.037877 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.036948    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-systemd-units\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037036    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwnhm\" (UniqueName: \"kubernetes.io/projected/69ed7a64-076f-4ecb-868e-78f46bb926dd-kube-api-access-rwnhm\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037090    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-hosts-file\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037710    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovnkube-config\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-os-release\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnktk\" (UniqueName: \"kubernetes.io/projected/98b238a4-272b-43c3-bcb0-b5cead6a44c0-kube-api-access-fnktk\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysconfig\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d285714b-fe13-43e7-b7e3-1e7011dd666b-tmp\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037846    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27fda661-b8a4-4bed-b977-ccf99ecc406e-serviceca\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-env-overrides\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdfxs\" (UniqueName: \"kubernetes.io/projected/1228f9c2-e269-4f64-a677-fae2b38ed3d7-kube-api-access-hdfxs\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cnibin\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.037998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-lib-modules\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038041    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-cni-multus\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-multus-certs\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.038662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41d3dc94-0044-4326-92be-d1550a0502b1-dbus\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.039438 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1228f9c2-e269-4f64-a677-fae2b38ed3d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.039438 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038120    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-registration-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.039438 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysctl-conf\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.039438 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038215    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-env-overrides\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.039438 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.038477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovnkube-config\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.040896 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.040875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1228f9c2-e269-4f64-a677-fae2b38ed3d7-ovn-node-metrics-cert\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.043175 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.043154    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bqv9f"
Jul 15 19:58:41.048652 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.048605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnfsc\" (UniqueName: \"kubernetes.io/projected/27fda661-b8a4-4bed-b977-ccf99ecc406e-kube-api-access-mnfsc\") pod \"node-ca-8z8br\" (UID: \"27fda661-b8a4-4bed-b977-ccf99ecc406e\") " pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:41.053434 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.053405    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdfxs\" (UniqueName: \"kubernetes.io/projected/1228f9c2-e269-4f64-a677-fae2b38ed3d7-kube-api-access-hdfxs\") pod \"ovnkube-node-vr9sl\" (UID: \"1228f9c2-e269-4f64-a677-fae2b38ed3d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.138697 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-cni-bin\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138697 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-hostroot\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysctl-d\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138741    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-var-lib-kubelet\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-cni-bin\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-k8s-cni-cncf-io\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138782    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-hostroot\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-etc-kubernetes\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-k8s-cni-cncf-io\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-etc-kubernetes\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n2hq\" (UniqueName: \"kubernetes.io/projected/31004923-49f0-49f5-9c4e-41abeffb6da6-kube-api-access-9n2hq\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-var-lib-kubelet\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.138926 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138963    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysctl-d\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.138995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-tuned\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-tmp-dir\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jfcr\" (UniqueName: \"kubernetes.io/projected/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-kube-api-access-9jfcr\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-cni-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-conf-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-run\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139170    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-host\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139183    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-conf-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41d3dc94-0044-4326-92be-d1550a0502b1-kubelet-config\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-cni-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-os-release\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-host\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139235    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-run\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.139502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-tmp-dir\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-os-release\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41d3dc94-0044-4326-92be-d1550a0502b1-kubelet-config\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f6563d9-f278-4f12-a251-c7808bbe1f85-agent-certs\") pod \"konnectivity-agent-bbmcl\" (UID: \"6f6563d9-f278-4f12-a251-c7808bbe1f85\") " pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31004923-49f0-49f5-9c4e-41abeffb6da6-cni-binary-copy\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtq5m\" (UniqueName: \"kubernetes.io/projected/6e1a0211-5b8b-4bf4-ae26-362b79b02546-kube-api-access-rtq5m\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98b238a4-272b-43c3-bcb0-b5cead6a44c0-iptables-alerter-script\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139619    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-kubernetes\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-socket-dir-parent\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98b238a4-272b-43c3-bcb0-b5cead6a44c0-host-slash\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-modprobe-d\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-kubernetes\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-sys-fs\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pczf7\" (UniqueName: \"kubernetes.io/projected/24c2d436-4147-4c2d-b062-77c339b3093c-kube-api-access-pczf7\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:41.140302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139817    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-sys-fs\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5rxv\" (UniqueName: \"kubernetes.io/projected/d285714b-fe13-43e7-b7e3-1e7011dd666b-kube-api-access-f5rxv\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98b238a4-272b-43c3-bcb0-b5cead6a44c0-host-slash\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139866    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-system-cni-dir\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-socket-dir-parent\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139886    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31004923-49f0-49f5-9c4e-41abeffb6da6-cni-binary-copy\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-socket-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139894    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98b238a4-272b-43c3-bcb0-b5cead6a44c0-iptables-alerter-script\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwnhm\" (UniqueName: \"kubernetes.io/projected/69ed7a64-076f-4ecb-868e-78f46bb926dd-kube-api-access-rwnhm\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-hosts-file\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139981    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-os-release\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-socket-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnktk\" (UniqueName: \"kubernetes.io/projected/98b238a4-272b-43c3-bcb0-b5cead6a44c0-kube-api-access-fnktk\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140078    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-hosts-file\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-system-cni-dir\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.139918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-modprobe-d\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141083 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysconfig\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140123    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysconfig\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d285714b-fe13-43e7-b7e3-1e7011dd666b-tmp\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140107    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-os-release\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cnibin\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cnibin\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140222    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-lib-modules\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-cni-multus\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-multus-certs\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41d3dc94-0044-4326-92be-d1550a0502b1-dbus\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140315    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-cni-multus\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-multus-certs\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-registration-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140354    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-lib-modules\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysctl-conf\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140375    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41d3dc94-0044-4326-92be-d1550a0502b1-dbus\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140402    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-cnibin\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.141866 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-registration-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140428    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-device-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-cnibin\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-kubelet\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-sysctl-conf\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/69ed7a64-076f-4ecb-868e-78f46bb926dd-device-dir\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-var-lib-kubelet\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f6563d9-f278-4f12-a251-c7808bbe1f85-konnectivity-ca\") pod \"konnectivity-agent-bbmcl\" (UID: \"6f6563d9-f278-4f12-a251-c7808bbe1f85\") " pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.140559    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140583    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-systemd\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-system-cni-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140661    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-netns\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140689    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-systemd\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.142747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-daemon-config\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.140756    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs podName:24c2d436-4147-4c2d-b062-77c339b3093c nodeName:}" failed. No retries permitted until 2026-07-15 19:58:41.640735174 +0000 UTC m=+3.089108263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs") pod "network-metrics-daemon-694gf" (UID: "24c2d436-4147-4c2d-b062-77c339b3093c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.140795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-host-run-netns\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.140863    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.140916    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret podName:41d3dc94-0044-4326-92be-d1550a0502b1 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:41.640902563 +0000 UTC m=+3.089275648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret") pod "global-pull-secret-syncer-zh2mc" (UID: "41d3dc94-0044-4326-92be-d1550a0502b1") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.141057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.141087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f6563d9-f278-4f12-a251-c7808bbe1f85-konnectivity-ca\") pod \"konnectivity-agent-bbmcl\" (UID: \"6f6563d9-f278-4f12-a251-c7808bbe1f85\") " pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.141097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-sys\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.141106    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31004923-49f0-49f5-9c4e-41abeffb6da6-system-cni-dir\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.141175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d285714b-fe13-43e7-b7e3-1e7011dd666b-sys\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.141195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31004923-49f0-49f5-9c4e-41abeffb6da6-multus-daemon-config\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.141320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1a0211-5b8b-4bf4-ae26-362b79b02546-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.142416    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f6563d9-f278-4f12-a251-c7808bbe1f85-agent-certs\") pod \"konnectivity-agent-bbmcl\" (UID: \"6f6563d9-f278-4f12-a251-c7808bbe1f85\") " pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.142470    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d285714b-fe13-43e7-b7e3-1e7011dd666b-tmp\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.143484 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.143075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d285714b-fe13-43e7-b7e3-1e7011dd666b-etc-tuned\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.149213 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.149196    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:58:41.149213 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.149219    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:58:41.149385 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.149232    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bmlw4 for pod openshift-network-diagnostics/network-check-target-ft87c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:41.149385 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.149285    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4 podName:afe79d63-635a-4f4d-9e66-d134b6ab4ba6 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:41.649267398 +0000 UTC m=+3.097640485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bmlw4" (UniqueName: "kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4") pod "network-check-target-ft87c" (UID: "afe79d63-635a-4f4d-9e66-d134b6ab4ba6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:41.151740 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.151712    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jfcr\" (UniqueName: \"kubernetes.io/projected/d2a9b2f0-1e6d-44d2-819f-3e09690c879f-kube-api-access-9jfcr\") pod \"node-resolver-lwq7z\" (UID: \"d2a9b2f0-1e6d-44d2-819f-3e09690c879f\") " pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.152332 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.152309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n2hq\" (UniqueName: \"kubernetes.io/projected/31004923-49f0-49f5-9c4e-41abeffb6da6-kube-api-access-9n2hq\") pod \"multus-b4zjt\" (UID: \"31004923-49f0-49f5-9c4e-41abeffb6da6\") " pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.152409 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.152332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pczf7\" (UniqueName: \"kubernetes.io/projected/24c2d436-4147-4c2d-b062-77c339b3093c-kube-api-access-pczf7\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:41.152744 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.152725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwnhm\" (UniqueName: \"kubernetes.io/projected/69ed7a64-076f-4ecb-868e-78f46bb926dd-kube-api-access-rwnhm\") pod \"aws-ebs-csi-driver-node-stg8f\" (UID: \"69ed7a64-076f-4ecb-868e-78f46bb926dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.153033 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.153013    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnktk\" (UniqueName: \"kubernetes.io/projected/98b238a4-272b-43c3-bcb0-b5cead6a44c0-kube-api-access-fnktk\") pod \"iptables-alerter-ksr9b\" (UID: \"98b238a4-272b-43c3-bcb0-b5cead6a44c0\") " pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.153106 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.153062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtq5m\" (UniqueName: \"kubernetes.io/projected/6e1a0211-5b8b-4bf4-ae26-362b79b02546-kube-api-access-rtq5m\") pod \"multus-additional-cni-plugins-j8dpm\" (UID: \"6e1a0211-5b8b-4bf4-ae26-362b79b02546\") " pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.153734 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.153713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5rxv\" (UniqueName: \"kubernetes.io/projected/d285714b-fe13-43e7-b7e3-1e7011dd666b-kube-api-access-f5rxv\") pod \"tuned-lvn92\" (UID: \"d285714b-fe13-43e7-b7e3-1e7011dd666b\") " pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.241798 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.241724    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:58:41.248745 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.248720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8z8br"
Jul 15 19:58:41.256858 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.256837    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b4zjt"
Jul 15 19:58:41.261379 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.261356    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ksr9b"
Jul 15 19:58:41.267871 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.267855    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j8dpm"
Jul 15 19:58:41.274359 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.274343    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:58:41.280879 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.280860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f"
Jul 15 19:58:41.287313 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.287291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lvn92"
Jul 15 19:58:41.291867 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.291850    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwq7z"
Jul 15 19:58:41.644201 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.644128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:41.644201 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.644193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:41.644388 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.644283    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:41.644388 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.644331    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs podName:24c2d436-4147-4c2d-b062-77c339b3093c nodeName:}" failed. No retries permitted until 2026-07-15 19:58:42.644316595 +0000 UTC m=+4.092689669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs") pod "network-metrics-daemon-694gf" (UID: "24c2d436-4147-4c2d-b062-77c339b3093c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:41.644388 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.644283    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:41.644520 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.644408    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret podName:41d3dc94-0044-4326-92be-d1550a0502b1 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:42.644393541 +0000 UTC m=+4.092766618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret") pod "global-pull-secret-syncer-zh2mc" (UID: "41d3dc94-0044-4326-92be-d1550a0502b1") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:41.745168 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:41.745135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:41.745344 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.745304    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:58:41.745344 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.745327    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:58:41.745344 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.745337    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bmlw4 for pod openshift-network-diagnostics/network-check-target-ft87c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:41.745446 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:41.745398    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4 podName:afe79d63-635a-4f4d-9e66-d134b6ab4ba6 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:42.74538212 +0000 UTC m=+4.193755194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bmlw4" (UniqueName: "kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4") pod "network-check-target-ft87c" (UID: "afe79d63-635a-4f4d-9e66-d134b6ab4ba6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:41.841805 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.841765    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ed7a64_076f_4ecb_868e_78f46bb926dd.slice/crio-ca19aaf6d926b535102d58255c8c4b944ba21806582eee7bccd680e8b88f2bb2 WatchSource:0}: Error finding container ca19aaf6d926b535102d58255c8c4b944ba21806582eee7bccd680e8b88f2bb2: Status 404 returned error can't find the container with id ca19aaf6d926b535102d58255c8c4b944ba21806582eee7bccd680e8b88f2bb2
Jul 15 19:58:41.842955 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.842927    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6563d9_f278_4f12_a251_c7808bbe1f85.slice/crio-f2dd149c660bd6e240a87cda41a94ad5a9f785161018b7666835120371e34b41 WatchSource:0}: Error finding container f2dd149c660bd6e240a87cda41a94ad5a9f785161018b7666835120371e34b41: Status 404 returned error can't find the container with id f2dd149c660bd6e240a87cda41a94ad5a9f785161018b7666835120371e34b41
Jul 15 19:58:41.843688 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.843638    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31004923_49f0_49f5_9c4e_41abeffb6da6.slice/crio-c5286a5521ecf6c570e366ee14c89a3bd371ec5ff4b86849f2640de63a7493cc WatchSource:0}: Error finding container c5286a5521ecf6c570e366ee14c89a3bd371ec5ff4b86849f2640de63a7493cc: Status 404 returned error can't find the container with id c5286a5521ecf6c570e366ee14c89a3bd371ec5ff4b86849f2640de63a7493cc
Jul 15 19:58:41.847044 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.847021    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fda661_b8a4_4bed_b977_ccf99ecc406e.slice/crio-3509eb4e1b05b24bd92e3bc5ddfd4e7daf44ea76ebba7c5caa8e254cd36c4126 WatchSource:0}: Error finding container 3509eb4e1b05b24bd92e3bc5ddfd4e7daf44ea76ebba7c5caa8e254cd36c4126: Status 404 returned error can't find the container with id 3509eb4e1b05b24bd92e3bc5ddfd4e7daf44ea76ebba7c5caa8e254cd36c4126
Jul 15 19:58:41.848225 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.848201    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1228f9c2_e269_4f64_a677_fae2b38ed3d7.slice/crio-0a17905fb6b8aa8042605af4c952129cd04b14739eb31589510965ead5bd734f WatchSource:0}: Error finding container 0a17905fb6b8aa8042605af4c952129cd04b14739eb31589510965ead5bd734f: Status 404 returned error can't find the container with id 0a17905fb6b8aa8042605af4c952129cd04b14739eb31589510965ead5bd734f
Jul 15 19:58:41.849002 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.848962    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a9b2f0_1e6d_44d2_819f_3e09690c879f.slice/crio-0071c979e3f85cfb499a3e1c0f9c36e5ae0c1b5e01ed8e47026a86cdc7c73fbd WatchSource:0}: Error finding container 0071c979e3f85cfb499a3e1c0f9c36e5ae0c1b5e01ed8e47026a86cdc7c73fbd: Status 404 returned error can't find the container with id 0071c979e3f85cfb499a3e1c0f9c36e5ae0c1b5e01ed8e47026a86cdc7c73fbd
Jul 15 19:58:41.849738 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.849719    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b238a4_272b_43c3_bcb0_b5cead6a44c0.slice/crio-fc0f89bb2666f6c4244952726208308708ddcd6e9b65a1a4f179aee5c269ed06 WatchSource:0}: Error finding container fc0f89bb2666f6c4244952726208308708ddcd6e9b65a1a4f179aee5c269ed06: Status 404 returned error can't find the container with id fc0f89bb2666f6c4244952726208308708ddcd6e9b65a1a4f179aee5c269ed06
Jul 15 19:58:41.851080 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.850993    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd285714b_fe13_43e7_b7e3_1e7011dd666b.slice/crio-9690b7e9d8ff9a25f447279aeb0896ede1c788ad1e38f1af9ae6ee8b2ec4174e WatchSource:0}: Error finding container 9690b7e9d8ff9a25f447279aeb0896ede1c788ad1e38f1af9ae6ee8b2ec4174e: Status 404 returned error can't find the container with id 9690b7e9d8ff9a25f447279aeb0896ede1c788ad1e38f1af9ae6ee8b2ec4174e
Jul 15 19:58:41.852308 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:58:41.852266    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1a0211_5b8b_4bf4_ae26_362b79b02546.slice/crio-63db1e7b14272a084075197328aa57857cb69eac8c84fa6745eec2d727d20905 WatchSource:0}: Error finding container 63db1e7b14272a084075197328aa57857cb69eac8c84fa6745eec2d727d20905: Status 404 returned error can't find the container with id 63db1e7b14272a084075197328aa57857cb69eac8c84fa6745eec2d727d20905
Jul 15 19:58:42.046693 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.046526    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:53:41 +0000 UTC" deadline="2028-01-15 21:37:46.004937119 +0000 UTC"
Jul 15 19:58:42.046693 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.046690    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13177h39m3.95825011s"
Jul 15 19:58:42.080728 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.080698    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerStarted","Data":"63db1e7b14272a084075197328aa57857cb69eac8c84fa6745eec2d727d20905"}
Jul 15 19:58:42.081659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.081633    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ksr9b" event={"ID":"98b238a4-272b-43c3-bcb0-b5cead6a44c0","Type":"ContainerStarted","Data":"fc0f89bb2666f6c4244952726208308708ddcd6e9b65a1a4f179aee5c269ed06"}
Jul 15 19:58:42.082692 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.082669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwq7z" event={"ID":"d2a9b2f0-1e6d-44d2-819f-3e09690c879f","Type":"ContainerStarted","Data":"0071c979e3f85cfb499a3e1c0f9c36e5ae0c1b5e01ed8e47026a86cdc7c73fbd"}
Jul 15 19:58:42.086746 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.086721    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8z8br" event={"ID":"27fda661-b8a4-4bed-b977-ccf99ecc406e","Type":"ContainerStarted","Data":"3509eb4e1b05b24bd92e3bc5ddfd4e7daf44ea76ebba7c5caa8e254cd36c4126"}
Jul 15 19:58:42.089182 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.089164    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b4zjt" event={"ID":"31004923-49f0-49f5-9c4e-41abeffb6da6","Type":"ContainerStarted","Data":"c5286a5521ecf6c570e366ee14c89a3bd371ec5ff4b86849f2640de63a7493cc"}
Jul 15 19:58:42.090113 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.090096    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f" event={"ID":"69ed7a64-076f-4ecb-868e-78f46bb926dd","Type":"ContainerStarted","Data":"ca19aaf6d926b535102d58255c8c4b944ba21806582eee7bccd680e8b88f2bb2"}
Jul 15 19:58:42.091615 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.091595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" event={"ID":"a3d8921e521c473a7998a09a95a39b65","Type":"ContainerStarted","Data":"54889dea89d3e77e206dc2b8bd8d69ca9ad06789c70505849cf00d3d6fdea979"}
Jul 15 19:58:42.092533 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.092515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lvn92" event={"ID":"d285714b-fe13-43e7-b7e3-1e7011dd666b","Type":"ContainerStarted","Data":"9690b7e9d8ff9a25f447279aeb0896ede1c788ad1e38f1af9ae6ee8b2ec4174e"}
Jul 15 19:58:42.093404 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.093385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bbmcl" event={"ID":"6f6563d9-f278-4f12-a251-c7808bbe1f85","Type":"ContainerStarted","Data":"f2dd149c660bd6e240a87cda41a94ad5a9f785161018b7666835120371e34b41"}
Jul 15 19:58:42.094726 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.094704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" event={"ID":"7f454b558f2d5d65067cce20f5bd5b71","Type":"ContainerStarted","Data":"b12cf8e90f7b7498f78ba01d190c57cfea492eea5c1c4bea5cf05633f9887740"}
Jul 15 19:58:42.095934 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.095913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"0a17905fb6b8aa8042605af4c952129cd04b14739eb31589510965ead5bd734f"}
Jul 15 19:58:42.108666 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.108630    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-52.ec2.internal" podStartSLOduration=2.108619706 podStartE2EDuration="2.108619706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:42.108499713 +0000 UTC m=+3.556872810" watchObservedRunningTime="2026-07-15 19:58:42.108619706 +0000 UTC m=+3.556992813"
Jul 15 19:58:42.654913 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.654086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:42.654913 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.654143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:42.654913 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.654309    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:42.654913 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.654369    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret podName:41d3dc94-0044-4326-92be-d1550a0502b1 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:44.654351561 +0000 UTC m=+6.102724638 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret") pod "global-pull-secret-syncer-zh2mc" (UID: "41d3dc94-0044-4326-92be-d1550a0502b1") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:42.654913 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.654777    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:42.654913 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.654826    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs podName:24c2d436-4147-4c2d-b062-77c339b3093c nodeName:}" failed. No retries permitted until 2026-07-15 19:58:44.654811174 +0000 UTC m=+6.103184250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs") pod "network-metrics-daemon-694gf" (UID: "24c2d436-4147-4c2d-b062-77c339b3093c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:42.755474 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:42.755427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:42.755647 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.755623    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:58:42.755647 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.755645    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:58:42.755757 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.755657    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bmlw4 for pod openshift-network-diagnostics/network-check-target-ft87c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:42.755757 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:42.755714    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4 podName:afe79d63-635a-4f4d-9e66-d134b6ab4ba6 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:44.755696331 +0000 UTC m=+6.204069419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bmlw4" (UniqueName: "kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4") pod "network-check-target-ft87c" (UID: "afe79d63-635a-4f4d-9e66-d134b6ab4ba6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:43.047186 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.047120    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:53:41 +0000 UTC" deadline="2028-03-17 19:30:56.194759629 +0000 UTC"
Jul 15 19:58:43.047186 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.047184    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14663h32m13.147580484s"
Jul 15 19:58:43.075308 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.075280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:43.075468 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:43.075432    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:43.075900 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.075879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:43.076015 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:43.075995    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:43.076076 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.076066    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:43.076166 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:43.076149    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:43.147054 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.146603    2562 generic.go:356] "Generic (PLEG): container finished" podID="7f454b558f2d5d65067cce20f5bd5b71" containerID="b12cf8e90f7b7498f78ba01d190c57cfea492eea5c1c4bea5cf05633f9887740" exitCode=0
Jul 15 19:58:43.147227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.147065    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" event={"ID":"7f454b558f2d5d65067cce20f5bd5b71","Type":"ContainerDied","Data":"b12cf8e90f7b7498f78ba01d190c57cfea492eea5c1c4bea5cf05633f9887740"}
Jul 15 19:58:43.147227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:43.147126    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" event={"ID":"7f454b558f2d5d65067cce20f5bd5b71","Type":"ContainerStarted","Data":"8c1e1e83db99e00b1c1a04877a6b5676b28cd5775ea128d7841eb8517b591db6"}
Jul 15 19:58:44.673990 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:44.673123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:44.673990 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:44.673235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:44.673990 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.673365    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:44.673990 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.673432    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs podName:24c2d436-4147-4c2d-b062-77c339b3093c nodeName:}" failed. No retries permitted until 2026-07-15 19:58:48.673411663 +0000 UTC m=+10.121784738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs") pod "network-metrics-daemon-694gf" (UID: "24c2d436-4147-4c2d-b062-77c339b3093c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:44.673990 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.673528    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:44.673990 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.673565    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret podName:41d3dc94-0044-4326-92be-d1550a0502b1 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:48.673553791 +0000 UTC m=+10.121926868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret") pod "global-pull-secret-syncer-zh2mc" (UID: "41d3dc94-0044-4326-92be-d1550a0502b1") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:44.774194 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:44.773522    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:44.774194 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.773742    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:58:44.774194 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.773763    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:58:44.774194 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.773775    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bmlw4 for pod openshift-network-diagnostics/network-check-target-ft87c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:44.774194 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:44.773831    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4 podName:afe79d63-635a-4f4d-9e66-d134b6ab4ba6 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:48.773812687 +0000 UTC m=+10.222185763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bmlw4" (UniqueName: "kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4") pod "network-check-target-ft87c" (UID: "afe79d63-635a-4f4d-9e66-d134b6ab4ba6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:45.073763 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:45.073733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:45.073942 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:45.073733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:45.073942 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:45.073870    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:45.073942 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:45.073898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:45.074122 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:45.074020    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:45.074122 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:45.074105    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:47.076758 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:47.076726    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:47.077220 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:47.076769    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:47.077220 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:47.076769    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:47.077220 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:47.076904    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:47.077387 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:47.077218    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:47.077387 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:47.077289    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:48.708509 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:48.708473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:48.708995 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:48.708526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:48.708995 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.708657    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:48.708995 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.708723    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:48.708995 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.708726    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret podName:41d3dc94-0044-4326-92be-d1550a0502b1 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:56.708705878 +0000 UTC m=+18.157078975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret") pod "global-pull-secret-syncer-zh2mc" (UID: "41d3dc94-0044-4326-92be-d1550a0502b1") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:48.708995 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.708806    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs podName:24c2d436-4147-4c2d-b062-77c339b3093c nodeName:}" failed. No retries permitted until 2026-07-15 19:58:56.708772671 +0000 UTC m=+18.157145749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs") pod "network-metrics-daemon-694gf" (UID: "24c2d436-4147-4c2d-b062-77c339b3093c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:48.809439 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:48.809397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:48.809597 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.809551    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:58:48.809597 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.809574    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:58:48.809597 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.809589    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bmlw4 for pod openshift-network-diagnostics/network-check-target-ft87c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:48.809753 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:48.809654    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4 podName:afe79d63-635a-4f4d-9e66-d134b6ab4ba6 nodeName:}" failed. No retries permitted until 2026-07-15 19:58:56.809633793 +0000 UTC m=+18.258006879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bmlw4" (UniqueName: "kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4") pod "network-check-target-ft87c" (UID: "afe79d63-635a-4f4d-9e66-d134b6ab4ba6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:49.074996 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:49.074521    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:49.074996 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:49.074645    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:49.074996 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:49.074684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:49.074996 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:49.074803    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:49.074996 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:49.074855    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:49.074996 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:49.074928    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:51.072861 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:51.072778    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:51.072861 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:51.072778    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:51.073375 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:51.072926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:51.073375 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:51.072921    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:51.073375 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:51.073072    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:51.073375 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:51.073172    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:53.072834 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:53.072796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:53.073328 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:53.072795    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:53.073328 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:53.072937    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:53.073328 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:53.072800    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:53.073328 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:53.073007    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:53.073328 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:53.073071    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:55.072802 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:55.072760    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:55.073269 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:55.072761    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:55.073269 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:55.072900    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:55.073269 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:55.072906    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:55.073269 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:55.073007    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:55.073269 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:55.073120    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:56.767897 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:56.767860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:56.767897 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:56.767901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:56.768547 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.768080    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:56.768547 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.768153    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:56.768547 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.768158    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs podName:24c2d436-4147-4c2d-b062-77c339b3093c nodeName:}" failed. No retries permitted until 2026-07-15 19:59:12.768137315 +0000 UTC m=+34.216510390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs") pod "network-metrics-daemon-694gf" (UID: "24c2d436-4147-4c2d-b062-77c339b3093c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:58:56.768547 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.768227    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret podName:41d3dc94-0044-4326-92be-d1550a0502b1 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:12.768207935 +0000 UTC m=+34.216581013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret") pod "global-pull-secret-syncer-zh2mc" (UID: "41d3dc94-0044-4326-92be-d1550a0502b1") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:58:56.868302 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:56.868272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:56.868496 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.868451    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:58:56.868496 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.868476    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:58:56.868496 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.868488    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bmlw4 for pod openshift-network-diagnostics/network-check-target-ft87c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:56.868709 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:56.868554    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4 podName:afe79d63-635a-4f4d-9e66-d134b6ab4ba6 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:12.868534188 +0000 UTC m=+34.316907270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bmlw4" (UniqueName: "kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4") pod "network-check-target-ft87c" (UID: "afe79d63-635a-4f4d-9e66-d134b6ab4ba6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:58:57.073247 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:57.073164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:57.073389 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:57.073165    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:57.073389 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:57.073274    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:58:57.073389 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:57.073164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:57.073389 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:57.073375    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:57.073592 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:57.073452    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:59.073875 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:59.073840    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:58:59.074357 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:59.074018    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:58:59.074357 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:59.074027    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:58:59.074357 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:59.074117    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:58:59.074357 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:58:59.074145    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:58:59.074357 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:58:59.074195    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:01.073365 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.072985    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:01.074050 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.072995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:01.074050 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:01.073476    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:01.074050 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.073049    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:01.074050 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:01.073584    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:01.074050 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:01.073657    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:01.194841 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.194812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8z8br" event={"ID":"27fda661-b8a4-4bed-b977-ccf99ecc406e","Type":"ContainerStarted","Data":"c09c3351bef87983582f142c03b64a4d45848835a3dcdbbe3dce7f4e043ed104"}
Jul 15 19:59:01.196210 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.196186    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b4zjt" event={"ID":"31004923-49f0-49f5-9c4e-41abeffb6da6","Type":"ContainerStarted","Data":"0e50d57ae3bf2f32e1adac434fe9010f6dfae8c616df8ae1e4f37201fee31057"}
Jul 15 19:59:01.197609 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.197582    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f" event={"ID":"69ed7a64-076f-4ecb-868e-78f46bb926dd","Type":"ContainerStarted","Data":"d6f48b0bc615320cffd2fb9c95c518f3a9b915ed9cb8b4fe0d97c5ab088e0567"}
Jul 15 19:59:01.199037 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.199014    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lvn92" event={"ID":"d285714b-fe13-43e7-b7e3-1e7011dd666b","Type":"ContainerStarted","Data":"fc67565d7642435bb7aeeed11ed97086bdae23cfd0603d731b2004d0f606afd2"}
Jul 15 19:59:01.200361 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.200341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bbmcl" event={"ID":"6f6563d9-f278-4f12-a251-c7808bbe1f85","Type":"ContainerStarted","Data":"95e5a069bfd1aa7fa4d65288bf646bd4c9b2dc8f35ac8e8bbb72fa92e0448f89"}
Jul 15 19:59:01.202716 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.202697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"17d9117d30f711990e3f83743e8c711a35aaa14b1edf08c74272c3c834b2cbfc"}
Jul 15 19:59:01.202815 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.202764    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"e4b38e978090666f4a6313d9440df40ee510f9d7d791d1ebc099382b8c3a7ae6"}
Jul 15 19:59:01.202873 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.202817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"cfb7f2390905e6c0aa8716bf1eaea3e3bb630dbc68a76537f1903d4b299d3915"}
Jul 15 19:59:01.202873 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.202868    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"a63a9c1a701e435f298c5fd29325f2258931ca70b3210966718161beaa064f6e"}
Jul 15 19:59:01.204194 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.204174    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e1a0211-5b8b-4bf4-ae26-362b79b02546" containerID="45eaf0ca2233065d4e635c12a8cc2e237cf191e74cefb80b6a9d867e6b179118" exitCode=0
Jul 15 19:59:01.204265 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.204232    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerDied","Data":"45eaf0ca2233065d4e635c12a8cc2e237cf191e74cefb80b6a9d867e6b179118"}
Jul 15 19:59:01.205562 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.205542    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwq7z" event={"ID":"d2a9b2f0-1e6d-44d2-819f-3e09690c879f","Type":"ContainerStarted","Data":"05a8fd7f34e36168dcdeae7f60b49c5939e38da6af765b2007059fa4f172c17e"}
Jul 15 19:59:01.215943 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.215905    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8z8br" podStartSLOduration=3.308183799 podStartE2EDuration="22.215893219s" totalImagesPullingTime="18.90770942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:01.215796954 +0000 UTC m=+22.664170050" watchObservedRunningTime="2026-07-15 19:59:01.215893219 +0000 UTC m=+22.664266313"
Jul 15 19:59:01.216048 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.215984    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-52.ec2.internal" podStartSLOduration=21.215977982 podStartE2EDuration="21.215977982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:58:43.164716005 +0000 UTC m=+4.613089103" watchObservedRunningTime="2026-07-15 19:59:01.215977982 +0000 UTC m=+22.664351069"
Jul 15 19:59:01.324611 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.324575    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:59:01.325086 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.325069    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:59:01.327316 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.327280    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bbmcl" podStartSLOduration=11.757743499 podStartE2EDuration="22.327267053s" totalImagesPullingTime="10.569523554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:01.295779032 +0000 UTC m=+22.744152128" watchObservedRunningTime="2026-07-15 19:59:01.327267053 +0000 UTC m=+22.775640173"
Jul 15 19:59:01.364868 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.364826    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lvn92" podStartSLOduration=3.459827758 podStartE2EDuration="22.364809933s" totalImagesPullingTime="18.904982175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:01.327629383 +0000 UTC m=+22.776002480" watchObservedRunningTime="2026-07-15 19:59:01.364809933 +0000 UTC m=+22.813183033"
Jul 15 19:59:01.401328 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.401285    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b4zjt" podStartSLOduration=3.367884158 podStartE2EDuration="22.401268989s" totalImagesPullingTime="19.033384831s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:01.364498953 +0000 UTC m=+22.812872061" watchObservedRunningTime="2026-07-15 19:59:01.401268989 +0000 UTC m=+22.849642085"
Jul 15 19:59:01.432814 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:01.432738    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lwq7z" podStartSLOduration=3.526558816 podStartE2EDuration="22.432719417s" totalImagesPullingTime="18.906160601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:01.401552779 +0000 UTC m=+22.849925876" watchObservedRunningTime="2026-07-15 19:59:01.432719417 +0000 UTC m=+22.881092513"
Jul 15 19:59:02.209005 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.208960    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ksr9b" event={"ID":"98b238a4-272b-43c3-bcb0-b5cead6a44c0","Type":"ContainerStarted","Data":"0fe176535d26a8ee71ce95a60f6a911f6681b45a2cb3fce1d65fbfc1d02b722c"}
Jul 15 19:59:02.212406 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.212379    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"60538622073cc83832f711519eb7cbcfc82a3b462889721fcda40b46375a9e2a"}
Jul 15 19:59:02.212533 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.212453    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"320798e16a2a11193892fe5b7350ea3c36f8aee0931777aa7b0575248c86551b"}
Jul 15 19:59:02.215453 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.215430    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:59:02.215843 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.215826    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bbmcl"
Jul 15 19:59:02.252571 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.252522    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ksr9b" podStartSLOduration=4.366421124 podStartE2EDuration="23.252501393s" totalImagesPullingTime="18.886080269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:02.23382352 +0000 UTC m=+23.682196616" watchObservedRunningTime="2026-07-15 19:59:02.252501393 +0000 UTC m=+23.700874491"
Jul 15 19:59:02.332156 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.332123    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 19:59:02.995211 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.995102    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:59:02.332150484Z","UUID":"9568c489-daed-4dd7-a726-d88e49bd3614","Handler":null,"Name":"","Endpoint":""}
Jul 15 19:59:02.997184 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.996855    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 19:59:02.997184 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:02.996893    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 19:59:03.073172 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:03.073142    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:03.073337 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:03.073150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:03.073337 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:03.073280    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:03.073464 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:03.073143    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:03.074001 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:03.073802    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:03.074673 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:03.074509    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:03.217114 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:03.217076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f" event={"ID":"69ed7a64-076f-4ecb-868e-78f46bb926dd","Type":"ContainerStarted","Data":"cd46d94ae272112d6ee2a29f7da1d97f3e5f6e50770cca79cd9f0238750194d4"}
Jul 15 19:59:03.985487 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:03.985444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmcl_6f6563d9-f278-4f12-a251-c7808bbe1f85/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:04.114127 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:04.114094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmcl_6f6563d9-f278-4f12-a251-c7808bbe1f85/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:04.221950 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:04.221856    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"df2fe2876bb1fde837d2c3f5108d6b19131f6eb08099cef1e66ceab9da2d1276"}
Jul 15 19:59:04.224142 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:04.224093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f" event={"ID":"69ed7a64-076f-4ecb-868e-78f46bb926dd","Type":"ContainerStarted","Data":"f6f29637f0e7654561ae7ef57130fe4c06d849befcd7f136a2b422d37abd0fb3"}
Jul 15 19:59:04.245657 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:04.245580    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stg8f" podStartSLOduration=4.174336189 podStartE2EDuration="25.245566917s" totalImagesPullingTime="21.071230728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:04.245062166 +0000 UTC m=+25.693435264" watchObservedRunningTime="2026-07-15 19:59:04.245566917 +0000 UTC m=+25.693940037"
Jul 15 19:59:05.072561 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:05.072527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:05.072743 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:05.072527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:05.072743 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:05.072644    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:05.072857 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:05.072737    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:05.072857 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:05.072527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:05.072857 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:05.072837    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:06.232021 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:06.231805    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" event={"ID":"1228f9c2-e269-4f64-a677-fae2b38ed3d7","Type":"ContainerStarted","Data":"5048010e3cc395a547642dc32a69065d8616bb6e0bd5a53e01fb1dc05eaeec9f"}
Jul 15 19:59:06.232935 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:06.232088    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:59:06.233783 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:06.233757    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e1a0211-5b8b-4bf4-ae26-362b79b02546" containerID="e8ded424b41f2ddbd83d30a8dc5977b5307b79480b36a9ecc500259b1473775f" exitCode=0
Jul 15 19:59:06.233924 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:06.233794    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerDied","Data":"e8ded424b41f2ddbd83d30a8dc5977b5307b79480b36a9ecc500259b1473775f"}
Jul 15 19:59:06.248600 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:06.248578    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:59:06.339767 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:06.339577    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl" podStartSLOduration=8.382070258 podStartE2EDuration="27.339560922s" totalImagesPullingTime="18.957490664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:06.300324323 +0000 UTC m=+27.748697419" watchObservedRunningTime="2026-07-15 19:59:06.339560922 +0000 UTC m=+27.787934018"
Jul 15 19:59:07.073377 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:07.073345    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:07.073579 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:07.073345    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:07.073579 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:07.073480    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:07.073579 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:07.073345    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:07.073734 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:07.073609    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:07.073734 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:07.073689    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:07.236233 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:07.236203    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:59:07.236674 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:07.236267    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:59:07.250492 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:07.250468    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:59:08.240412 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.240379    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e1a0211-5b8b-4bf4-ae26-362b79b02546" containerID="01f15dd373df8f5d3f196ad891a2b75fc9116fabf84aaa9d35ed81579af91f18" exitCode=0
Jul 15 19:59:08.241145 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.240442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerDied","Data":"01f15dd373df8f5d3f196ad891a2b75fc9116fabf84aaa9d35ed81579af91f18"}
Jul 15 19:59:08.304090 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.304054    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-694gf"]
Jul 15 19:59:08.304247 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.304189    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:08.304333 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:08.304313    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:08.323917 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.323847    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ft87c"]
Jul 15 19:59:08.324074 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.323953    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:08.324074 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:08.324060    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:08.324453 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.324429    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zh2mc"]
Jul 15 19:59:08.324576 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:08.324528    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:08.324632 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:08.324608    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:10.073430 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:10.073355    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:10.073796 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:10.073457    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:10.073796 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:10.073357    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:10.073796 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:10.073477    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:10.073796 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:10.073583    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:10.073796 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:10.073646    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:11.248904 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:11.248644    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e1a0211-5b8b-4bf4-ae26-362b79b02546" containerID="df6a720b1f27b447c89fb6be8be5165231b7dfe95aa5ab46eb408ec36771d557" exitCode=0
Jul 15 19:59:11.248904 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:11.248721    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerDied","Data":"df6a720b1f27b447c89fb6be8be5165231b7dfe95aa5ab46eb408ec36771d557"}
Jul 15 19:59:12.073226 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:12.073190    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:12.073415 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:12.073241    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:12.073415 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:12.073190    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:12.073415 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.073337    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:12.073598 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.073459    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:12.073598 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.073519    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:12.780113 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:12.780082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:12.780554 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:12.780122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:12.780554 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.780240    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:59:12.780554 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.780251    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:59:12.780554 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.780303    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret podName:41d3dc94-0044-4326-92be-d1550a0502b1 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:44.780283666 +0000 UTC m=+66.228656743 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret") pod "global-pull-secret-syncer-zh2mc" (UID: "41d3dc94-0044-4326-92be-d1550a0502b1") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:59:12.780554 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.780323    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs podName:24c2d436-4147-4c2d-b062-77c339b3093c nodeName:}" failed. No retries permitted until 2026-07-15 19:59:44.780313558 +0000 UTC m=+66.228686636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs") pod "network-metrics-daemon-694gf" (UID: "24c2d436-4147-4c2d-b062-77c339b3093c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:59:12.881219 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:12.881183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:12.881408 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.881358    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:59:12.881408 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.881380    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:59:12.881408 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.881390    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bmlw4 for pod openshift-network-diagnostics/network-check-target-ft87c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:59:12.881554 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:12.881455    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4 podName:afe79d63-635a-4f4d-9e66-d134b6ab4ba6 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:44.881436309 +0000 UTC m=+66.329809396 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bmlw4" (UniqueName: "kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4") pod "network-check-target-ft87c" (UID: "afe79d63-635a-4f4d-9e66-d134b6ab4ba6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:59:14.072978 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.072931    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:14.072978 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.072956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:14.073383 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.072956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:14.073383 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.073102    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zh2mc" podUID="41d3dc94-0044-4326-92be-d1550a0502b1"
Jul 15 19:59:14.073383 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.073211    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-694gf" podUID="24c2d436-4147-4c2d-b062-77c339b3093c"
Jul 15 19:59:14.073383 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.073288    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ft87c" podUID="afe79d63-635a-4f4d-9e66-d134b6ab4ba6"
Jul 15 19:59:14.361680 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.361603    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-52.ec2.internal" event="NodeReady"
Jul 15 19:59:14.361845 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.361760    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 19:59:14.419714 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.419676    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-89dc4f459-k4gtm"]
Jul 15 19:59:14.422446 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.422420    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.425706 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.425683    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 19:59:14.425833 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.425730    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqj2h\""
Jul 15 19:59:14.426077 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.426045    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 19:59:14.426077 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.426069    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 19:59:14.431647 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.431627    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 19:59:14.434427 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.434407    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-54fvs"]
Jul 15 19:59:14.436312 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.436294    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xsqkj"]
Jul 15 19:59:14.436451 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.436430    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.437867 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.437851    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:14.439657 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.439460    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wppjw\""
Jul 15 19:59:14.440608 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.439823    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 19:59:14.440608 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.440259    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 19:59:14.440608 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.440332    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-89dc4f459-k4gtm"]
Jul 15 19:59:14.440608 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.440476    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 19:59:14.440608 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.440497    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tvpz5\""
Jul 15 19:59:14.440909 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.440745    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 19:59:14.441059 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.441037    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 19:59:14.450095 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.450057    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xsqkj"]
Jul 15 19:59:14.454260 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.454238    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-54fvs"]
Jul 15 19:59:14.595603 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595570    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd9fdcd-774e-495b-888f-ddfe727138bf-tmp-dir\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.595766 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595620    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.595766 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-ca-trust-extracted\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.595766 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.595766 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595729    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bd9fdcd-774e-495b-888f-ddfe727138bf-config-volume\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.595766 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-image-registry-private-configuration\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.595989 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-certificates\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.595989 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595794    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-bound-sa-token\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.595989 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a756270b-4ef2-4628-a1b4-eed8f5b25d3f-cert\") pod \"ingress-canary-xsqkj\" (UID: \"a756270b-4ef2-4628-a1b4-eed8f5b25d3f\") " pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:14.595989 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595928    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-installation-pull-secrets\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.595989 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.595955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqjnw\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-kube-api-access-kqjnw\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.596177 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.596001    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4bcc\" (UniqueName: \"kubernetes.io/projected/a756270b-4ef2-4628-a1b4-eed8f5b25d3f-kube-api-access-v4bcc\") pod \"ingress-canary-xsqkj\" (UID: \"a756270b-4ef2-4628-a1b4-eed8f5b25d3f\") " pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:14.596177 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.596016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-trusted-ca\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.596177 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.596059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9pmk\" (UniqueName: \"kubernetes.io/projected/3bd9fdcd-774e-495b-888f-ddfe727138bf-kube-api-access-k9pmk\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.696549 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9pmk\" (UniqueName: \"kubernetes.io/projected/3bd9fdcd-774e-495b-888f-ddfe727138bf-kube-api-access-k9pmk\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.696549 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd9fdcd-774e-495b-888f-ddfe727138bf-tmp-dir\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.696757 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.696757 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-ca-trust-extracted\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.696757 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.696727    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:59:14.696757 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.696747    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89dc4f459-k4gtm: secret "image-registry-tls" not found
Jul 15 19:59:14.696945 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.696945 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.696817    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls podName:11f8b0cf-010e-4a3d-8e2b-4e6e3add672e nodeName:}" failed. No retries permitted until 2026-07-15 19:59:15.196793681 +0000 UTC m=+36.645166777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls") pod "image-registry-89dc4f459-k4gtm" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e") : secret "image-registry-tls" not found
Jul 15 19:59:14.696945 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bd9fdcd-774e-495b-888f-ddfe727138bf-config-volume\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.696945 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696903    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-image-registry-private-configuration\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.696945 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd9fdcd-774e-495b-888f-ddfe727138bf-tmp-dir\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.696940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-certificates\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.697022    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697044    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-ca-trust-extracted\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:14.697062    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls podName:3bd9fdcd-774e-495b-888f-ddfe727138bf nodeName:}" failed. No retries permitted until 2026-07-15 19:59:15.197048702 +0000 UTC m=+36.645421779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls") pod "dns-default-54fvs" (UID: "3bd9fdcd-774e-495b-888f-ddfe727138bf") : secret "dns-default-metrics-tls" not found
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-bound-sa-token\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a756270b-4ef2-4628-a1b4-eed8f5b25d3f-cert\") pod \"ingress-canary-xsqkj\" (UID: \"a756270b-4ef2-4628-a1b4-eed8f5b25d3f\") " pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697146    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-installation-pull-secrets\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjnw\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-kube-api-access-kqjnw\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.697227 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bcc\" (UniqueName: \"kubernetes.io/projected/a756270b-4ef2-4628-a1b4-eed8f5b25d3f-kube-api-access-v4bcc\") pod \"ingress-canary-xsqkj\" (UID: \"a756270b-4ef2-4628-a1b4-eed8f5b25d3f\") " pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:14.697646 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-trusted-ca\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.697646 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697519    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-certificates\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.697646 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.697599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bd9fdcd-774e-495b-888f-ddfe727138bf-config-volume\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.698524 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.698504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-trusted-ca\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.701605 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.701581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-image-registry-private-configuration\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.701605 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.701603    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a756270b-4ef2-4628-a1b4-eed8f5b25d3f-cert\") pod \"ingress-canary-xsqkj\" (UID: \"a756270b-4ef2-4628-a1b4-eed8f5b25d3f\") " pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:14.701744 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.701606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-installation-pull-secrets\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.708990 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.706297    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9pmk\" (UniqueName: \"kubernetes.io/projected/3bd9fdcd-774e-495b-888f-ddfe727138bf-kube-api-access-k9pmk\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:14.708990 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.706322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-bound-sa-token\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.710993 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.710953    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqjnw\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-kube-api-access-kqjnw\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:14.711315 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.711296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4bcc\" (UniqueName: \"kubernetes.io/projected/a756270b-4ef2-4628-a1b4-eed8f5b25d3f-kube-api-access-v4bcc\") pod \"ingress-canary-xsqkj\" (UID: \"a756270b-4ef2-4628-a1b4-eed8f5b25d3f\") " pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:14.753741 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:14.753704    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xsqkj"
Jul 15 19:59:15.201719 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:15.201682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:15.202428 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:15.201730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:15.202428 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:15.201852    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:59:15.202428 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:15.201875    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89dc4f459-k4gtm: secret "image-registry-tls" not found
Jul 15 19:59:15.202428 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:15.201909    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:59:15.202428 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:15.201945    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls podName:11f8b0cf-010e-4a3d-8e2b-4e6e3add672e nodeName:}" failed. No retries permitted until 2026-07-15 19:59:16.201923063 +0000 UTC m=+37.650296154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls") pod "image-registry-89dc4f459-k4gtm" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e") : secret "image-registry-tls" not found
Jul 15 19:59:15.202428 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:15.201963    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls podName:3bd9fdcd-774e-495b-888f-ddfe727138bf nodeName:}" failed. No retries permitted until 2026-07-15 19:59:16.201955004 +0000 UTC m=+37.650328091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls") pod "dns-default-54fvs" (UID: "3bd9fdcd-774e-495b-888f-ddfe727138bf") : secret "dns-default-metrics-tls" not found
Jul 15 19:59:15.696507 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:15.696249    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xsqkj"]
Jul 15 19:59:15.699496 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:15.699473    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda756270b_4ef2_4628_a1b4_eed8f5b25d3f.slice/crio-72e1d3adf0a6fc75891cea448b9bee51737387e6c51add7ded1b4bb9bab27033 WatchSource:0}: Error finding container 72e1d3adf0a6fc75891cea448b9bee51737387e6c51add7ded1b4bb9bab27033: Status 404 returned error can't find the container with id 72e1d3adf0a6fc75891cea448b9bee51737387e6c51add7ded1b4bb9bab27033
Jul 15 19:59:16.072938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.072905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:16.072938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.072934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:16.072938 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.072942    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:16.077048 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.076884    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:59:16.077048 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.076895    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:59:16.077048 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.076929    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kl6c8\""
Jul 15 19:59:16.077048 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.076929    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:59:16.077048 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.076949    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l6s2m\""
Jul 15 19:59:16.077048 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.076933    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:59:16.210845 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.210807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:16.211395 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.210851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:16.211395 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:16.211010    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:59:16.211395 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:16.211068    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls podName:3bd9fdcd-774e-495b-888f-ddfe727138bf nodeName:}" failed. No retries permitted until 2026-07-15 19:59:18.211054081 +0000 UTC m=+39.659427155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls") pod "dns-default-54fvs" (UID: "3bd9fdcd-774e-495b-888f-ddfe727138bf") : secret "dns-default-metrics-tls" not found
Jul 15 19:59:16.211395 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:16.211013    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:59:16.211395 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:16.211099    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89dc4f459-k4gtm: secret "image-registry-tls" not found
Jul 15 19:59:16.211395 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:16.211147    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls podName:11f8b0cf-010e-4a3d-8e2b-4e6e3add672e nodeName:}" failed. No retries permitted until 2026-07-15 19:59:18.211135626 +0000 UTC m=+39.659508704 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls") pod "image-registry-89dc4f459-k4gtm" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e") : secret "image-registry-tls" not found
Jul 15 19:59:16.264114 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.264083    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e1a0211-5b8b-4bf4-ae26-362b79b02546" containerID="aeb0f34775f847c522b23f2c26b60469a8af06fc0baead266a34d038a12930d4" exitCode=0
Jul 15 19:59:16.264298 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.264178    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerDied","Data":"aeb0f34775f847c522b23f2c26b60469a8af06fc0baead266a34d038a12930d4"}
Jul 15 19:59:16.265648 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:16.265611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xsqkj" event={"ID":"a756270b-4ef2-4628-a1b4-eed8f5b25d3f","Type":"ContainerStarted","Data":"72e1d3adf0a6fc75891cea448b9bee51737387e6c51add7ded1b4bb9bab27033"}
Jul 15 19:59:17.271902 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:17.271864    2562 generic.go:356] "Generic (PLEG): container finished" podID="6e1a0211-5b8b-4bf4-ae26-362b79b02546" containerID="e015d2eb6adf088c9dbd2057ecfde5d5ebc724cd338d72a364179802a9f45e27" exitCode=0
Jul 15 19:59:17.272397 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:17.271926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerDied","Data":"e015d2eb6adf088c9dbd2057ecfde5d5ebc724cd338d72a364179802a9f45e27"}
Jul 15 19:59:18.228821 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:18.228790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:18.228947 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:18.228830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:18.228947 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:18.228931    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:59:18.228947 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:18.228935    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:59:18.229071 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:18.228952    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89dc4f459-k4gtm: secret "image-registry-tls" not found
Jul 15 19:59:18.229071 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:18.229014    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls podName:11f8b0cf-010e-4a3d-8e2b-4e6e3add672e nodeName:}" failed. No retries permitted until 2026-07-15 19:59:22.228993837 +0000 UTC m=+43.677366927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls") pod "image-registry-89dc4f459-k4gtm" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e") : secret "image-registry-tls" not found
Jul 15 19:59:18.229071 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:18.229027    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls podName:3bd9fdcd-774e-495b-888f-ddfe727138bf nodeName:}" failed. No retries permitted until 2026-07-15 19:59:22.229021418 +0000 UTC m=+43.677394492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls") pod "dns-default-54fvs" (UID: "3bd9fdcd-774e-495b-888f-ddfe727138bf") : secret "dns-default-metrics-tls" not found
Jul 15 19:59:18.278896 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:18.278865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" event={"ID":"6e1a0211-5b8b-4bf4-ae26-362b79b02546","Type":"ContainerStarted","Data":"9606824c0413a4a9113cfe6b9e68bfe9e46bb0e8dfa3928296ba7f9ce9fe6fe5"}
Jul 15 19:59:18.306119 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:18.305916    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j8dpm" podStartSLOduration=8.261133386000001 podStartE2EDuration="39.305894726s" totalImagesPullingTime="31.04476134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:18.304049596 +0000 UTC m=+39.752422692" watchObservedRunningTime="2026-07-15 19:59:18.305894726 +0000 UTC m=+39.754267825"
Jul 15 19:59:19.282996 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:19.282704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xsqkj" event={"ID":"a756270b-4ef2-4628-a1b4-eed8f5b25d3f","Type":"ContainerStarted","Data":"a4a998ddf7530dbbb09660b37cbae09caddf2c4868648a81dc3ede516bc4e93a"}
Jul 15 19:59:19.300982 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:19.300916    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xsqkj" podStartSLOduration=2.780699722 podStartE2EDuration="5.300898116s" totalImagesPullingTime="2.520198394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:19.30020976 +0000 UTC m=+40.748582857" watchObservedRunningTime="2026-07-15 19:59:19.300898116 +0000 UTC m=+40.749271190"
Jul 15 19:59:20.985641 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:20.985609    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2vxzc"]
Jul 15 19:59:21.013718 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.013689    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2vxzc"]
Jul 15 19:59:21.013861 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.013820    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.018525 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.018505    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 19:59:21.018525 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.018504    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 19:59:21.019262 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.019241    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 19:59:21.019262 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.019257    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v5glv\""
Jul 15 19:59:21.019428 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.019250    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 19:59:21.149437 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.149405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d2f159-5f80-4228-bedd-8d261a02e1f4-data-volume\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.149597 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.149447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f3d2f159-5f80-4228-bedd-8d261a02e1f4-crio-socket\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.149597 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.149465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llgqd\" (UniqueName: \"kubernetes.io/projected/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-api-access-llgqd\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.149597 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.149493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.149597 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.149517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.250288 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.250208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d2f159-5f80-4228-bedd-8d261a02e1f4-data-volume\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.250288 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.250250    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f3d2f159-5f80-4228-bedd-8d261a02e1f4-crio-socket\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.250288 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.250272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llgqd\" (UniqueName: \"kubernetes.io/projected/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-api-access-llgqd\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.250544 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.250301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.250544 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.250326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.250544 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.250389    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:59:21.250544 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.250475    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:21.250544 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.250486    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:21.750467388 +0000 UTC m=+43.198840462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : configmap "kube-rbac-proxy" not found
Jul 15 19:59:21.250544 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.250498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f3d2f159-5f80-4228-bedd-8d261a02e1f4-crio-socket\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.250743 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.250558    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:21.750543266 +0000 UTC m=+43.198916342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:21.250743 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.250593    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d2f159-5f80-4228-bedd-8d261a02e1f4-data-volume\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.261255 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.261231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llgqd\" (UniqueName: \"kubernetes.io/projected/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-api-access-llgqd\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.755344 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.755310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.755521 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:21.755347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:21.755521 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.755457    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:59:21.755521 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.755511    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:21.755615 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.755539    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:22.755521758 +0000 UTC m=+44.203894831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : configmap "kube-rbac-proxy" not found
Jul 15 19:59:21.755653 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:21.755625    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:22.755600673 +0000 UTC m=+44.203973750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:22.259020 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:22.258986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:22.259020 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:22.259022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:22.259522 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.259162    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:59:22.259522 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.259184    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89dc4f459-k4gtm: secret "image-registry-tls" not found
Jul 15 19:59:22.259522 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.259194    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:59:22.259522 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.259252    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls podName:11f8b0cf-010e-4a3d-8e2b-4e6e3add672e nodeName:}" failed. No retries permitted until 2026-07-15 19:59:30.259231667 +0000 UTC m=+51.707604757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls") pod "image-registry-89dc4f459-k4gtm" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e") : secret "image-registry-tls" not found
Jul 15 19:59:22.259522 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.259270    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls podName:3bd9fdcd-774e-495b-888f-ddfe727138bf nodeName:}" failed. No retries permitted until 2026-07-15 19:59:30.259261679 +0000 UTC m=+51.707634753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls") pod "dns-default-54fvs" (UID: "3bd9fdcd-774e-495b-888f-ddfe727138bf") : secret "dns-default-metrics-tls" not found
Jul 15 19:59:22.726439 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:22.726410    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwq7z_d2a9b2f0-1e6d-44d2-819f-3e09690c879f/dns-node-resolver/0.log"
Jul 15 19:59:22.762829 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:22.762801    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:22.762896 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:22.762832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:22.762937 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.762921    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:59:22.763024 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.763004    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:24.762963145 +0000 UTC m=+46.211336219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : configmap "kube-rbac-proxy" not found
Jul 15 19:59:22.763111 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.763090    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:22.763165 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:22.763156    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:24.763139622 +0000 UTC m=+46.211512696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:23.928951 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:23.928921    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8z8br_27fda661-b8a4-4bed-b977-ccf99ecc406e/node-ca/0.log"
Jul 15 19:59:24.780615 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:24.780586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:24.780615 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:24.780617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:24.780808 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:24.780711    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:59:24.780808 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:24.780748    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:24.780808 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:24.780774    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.78075885 +0000 UTC m=+50.229131923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : configmap "kube-rbac-proxy" not found
Jul 15 19:59:24.780808 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:24.780790    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:28.780779177 +0000 UTC m=+50.229152250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:24.931249 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:24.931219    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xsqkj_a756270b-4ef2-4628-a1b4-eed8f5b25d3f/serve-healthcheck-canary/0.log"
Jul 15 19:59:28.808862 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:28.808826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:28.808862 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:28.808863    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:28.809323 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:28.808937    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:59:28.809323 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:28.808957    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:28.809323 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:28.809017    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:36.809000519 +0000 UTC m=+58.257373596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : configmap "kube-rbac-proxy" not found
Jul 15 19:59:28.809323 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:28.809079    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:36.809072302 +0000 UTC m=+58.257445376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:30.321540 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:30.321501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:30.321540 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:30.321539    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:30.322015 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:30.321656    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 19:59:30.322015 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:30.321672    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 19:59:30.322015 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:30.321693    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-89dc4f459-k4gtm: secret "image-registry-tls" not found
Jul 15 19:59:30.322015 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:30.321716    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls podName:3bd9fdcd-774e-495b-888f-ddfe727138bf nodeName:}" failed. No retries permitted until 2026-07-15 19:59:46.321697271 +0000 UTC m=+67.770070344 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls") pod "dns-default-54fvs" (UID: "3bd9fdcd-774e-495b-888f-ddfe727138bf") : secret "dns-default-metrics-tls" not found
Jul 15 19:59:30.322015 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:30.321746    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls podName:11f8b0cf-010e-4a3d-8e2b-4e6e3add672e nodeName:}" failed. No retries permitted until 2026-07-15 19:59:46.321729074 +0000 UTC m=+67.770102161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls") pod "image-registry-89dc4f459-k4gtm" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e") : secret "image-registry-tls" not found
Jul 15 19:59:36.873297 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:36.873259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:36.873297 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:36.873299    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:36.873851 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:36.873458    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:36.873851 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:36.873533    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls podName:f3d2f159-5f80-4228-bedd-8d261a02e1f4 nodeName:}" failed. No retries permitted until 2026-07-15 19:59:52.873512042 +0000 UTC m=+74.321885133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vxzc" (UID: "f3d2f159-5f80-4228-bedd-8d261a02e1f4") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:59:36.873987 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:36.873943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f3d2f159-5f80-4228-bedd-8d261a02e1f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:39.262331 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:39.262307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vr9sl"
Jul 15 19:59:44.830502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.830455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:44.830502 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.830507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:44.833533 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.833512    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:59:44.834662 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.834630    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:59:44.843440 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.843412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2d436-4147-4c2d-b062-77c339b3093c-metrics-certs\") pod \"network-metrics-daemon-694gf\" (UID: \"24c2d436-4147-4c2d-b062-77c339b3093c\") " pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:44.843440 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.843426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41d3dc94-0044-4326-92be-d1550a0502b1-original-pull-secret\") pod \"global-pull-secret-syncer-zh2mc\" (UID: \"41d3dc94-0044-4326-92be-d1550a0502b1\") " pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:44.888322 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.888302    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zh2mc"
Jul 15 19:59:44.896229 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.896209    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kl6c8\""
Jul 15 19:59:44.904020 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.903995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-694gf"
Jul 15 19:59:44.931133 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.931109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:44.936431 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.936410    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:59:44.944998 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.944823    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:59:44.955217 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:44.955068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmlw4\" (UniqueName: \"kubernetes.io/projected/afe79d63-635a-4f4d-9e66-d134b6ab4ba6-kube-api-access-bmlw4\") pod \"network-check-target-ft87c\" (UID: \"afe79d63-635a-4f4d-9e66-d134b6ab4ba6\") " pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:45.028339 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.028309    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zh2mc"]
Jul 15 19:59:45.031386 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:45.031358    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d3dc94_0044_4326_92be_d1550a0502b1.slice/crio-bab38a4350f0b432c43cb0c0593c3ea228830cbe2e89f64e576e2ecc3f6039fb WatchSource:0}: Error finding container bab38a4350f0b432c43cb0c0593c3ea228830cbe2e89f64e576e2ecc3f6039fb: Status 404 returned error can't find the container with id bab38a4350f0b432c43cb0c0593c3ea228830cbe2e89f64e576e2ecc3f6039fb
Jul 15 19:59:45.037881 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.037863    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-694gf"]
Jul 15 19:59:45.040462 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:45.040440    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c2d436_4147_4c2d_b062_77c339b3093c.slice/crio-ae9eb1e7929e49c3904a2567c1a7fa12bbd9cf3c9bccd4bb360603e1dfbde55c WatchSource:0}: Error finding container ae9eb1e7929e49c3904a2567c1a7fa12bbd9cf3c9bccd4bb360603e1dfbde55c: Status 404 returned error can't find the container with id ae9eb1e7929e49c3904a2567c1a7fa12bbd9cf3c9bccd4bb360603e1dfbde55c
Jul 15 19:59:45.185197 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.185179    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l6s2m\""
Jul 15 19:59:45.192701 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.192686    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:45.309114 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.309087    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ft87c"]
Jul 15 19:59:45.311958 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:45.311929    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe79d63_635a_4f4d_9e66_d134b6ab4ba6.slice/crio-baf82ac13e430b39df742ab94363823c165051769fa9fd025a7406d1894d4355 WatchSource:0}: Error finding container baf82ac13e430b39df742ab94363823c165051769fa9fd025a7406d1894d4355: Status 404 returned error can't find the container with id baf82ac13e430b39df742ab94363823c165051769fa9fd025a7406d1894d4355
Jul 15 19:59:45.345021 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.344995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zh2mc" event={"ID":"41d3dc94-0044-4326-92be-d1550a0502b1","Type":"ContainerStarted","Data":"bab38a4350f0b432c43cb0c0593c3ea228830cbe2e89f64e576e2ecc3f6039fb"}
Jul 15 19:59:45.346006 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.345983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ft87c" event={"ID":"afe79d63-635a-4f4d-9e66-d134b6ab4ba6","Type":"ContainerStarted","Data":"baf82ac13e430b39df742ab94363823c165051769fa9fd025a7406d1894d4355"}
Jul 15 19:59:45.347032 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:45.347013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-694gf" event={"ID":"24c2d436-4147-4c2d-b062-77c339b3093c","Type":"ContainerStarted","Data":"ae9eb1e7929e49c3904a2567c1a7fa12bbd9cf3c9bccd4bb360603e1dfbde55c"}
Jul 15 19:59:46.342102 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.342066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:46.342545 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.342116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:46.346019 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.345940    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd9fdcd-774e-495b-888f-ddfe727138bf-metrics-tls\") pod \"dns-default-54fvs\" (UID: \"3bd9fdcd-774e-495b-888f-ddfe727138bf\") " pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:46.349665 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.349620    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"image-registry-89dc4f459-k4gtm\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") " pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:46.536112 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.536083    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqj2h\""
Jul 15 19:59:46.543525 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.543495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:46.550743 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.550717    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wppjw\""
Jul 15 19:59:46.558218 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.558182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:46.728798 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.728752    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-89dc4f459-k4gtm"]
Jul 15 19:59:46.738713 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:46.738662    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-54fvs"]
Jul 15 19:59:46.869453 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:46.869380    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f8b0cf_010e_4a3d_8e2b_4e6e3add672e.slice/crio-722c09b67da0eb3c0e22796ae113a290175fc4c59944041cedfdbc3409c2188e WatchSource:0}: Error finding container 722c09b67da0eb3c0e22796ae113a290175fc4c59944041cedfdbc3409c2188e: Status 404 returned error can't find the container with id 722c09b67da0eb3c0e22796ae113a290175fc4c59944041cedfdbc3409c2188e
Jul 15 19:59:46.870411 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:46.870385    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd9fdcd_774e_495b_888f_ddfe727138bf.slice/crio-6483db16f67127c469e1758eab2f7a60e7f51d397575a8214b7d3f0e3f8cd882 WatchSource:0}: Error finding container 6483db16f67127c469e1758eab2f7a60e7f51d397575a8214b7d3f0e3f8cd882: Status 404 returned error can't find the container with id 6483db16f67127c469e1758eab2f7a60e7f51d397575a8214b7d3f0e3f8cd882
Jul 15 19:59:47.356197 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.356121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54fvs" event={"ID":"3bd9fdcd-774e-495b-888f-ddfe727138bf","Type":"ContainerStarted","Data":"6483db16f67127c469e1758eab2f7a60e7f51d397575a8214b7d3f0e3f8cd882"}
Jul 15 19:59:47.358798 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.358768    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-694gf" event={"ID":"24c2d436-4147-4c2d-b062-77c339b3093c","Type":"ContainerStarted","Data":"0a573ff24135c3dd445b0b0ceb5f837496f27aea42db96259bc37d6dde196c54"}
Jul 15 19:59:47.358931 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.358822    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-694gf" event={"ID":"24c2d436-4147-4c2d-b062-77c339b3093c","Type":"ContainerStarted","Data":"2d7fa8dcd1437c8f72fea4923d61580b31261fb8711f8a8a0856a633b9f0bbfd"}
Jul 15 19:59:47.360754 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.360726    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm" event={"ID":"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e","Type":"ContainerStarted","Data":"493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00"}
Jul 15 19:59:47.360858 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.360773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm" event={"ID":"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e","Type":"ContainerStarted","Data":"722c09b67da0eb3c0e22796ae113a290175fc4c59944041cedfdbc3409c2188e"}
Jul 15 19:59:47.361078 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.361032    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 19:59:47.400718 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.400344    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-694gf" podStartSLOduration=66.545432975 podStartE2EDuration="1m8.400324234s" totalImagesPullingTime="1.854891259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:47.399765157 +0000 UTC m=+68.848138252" watchObservedRunningTime="2026-07-15 19:59:47.400324234 +0000 UTC m=+68.848697333"
Jul 15 19:59:47.443573 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.442865    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm" podStartSLOduration=35.442844194 podStartE2EDuration="35.442844194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:47.441958242 +0000 UTC m=+68.890331339" watchObservedRunningTime="2026-07-15 19:59:47.442844194 +0000 UTC m=+68.891217292"
Jul 15 19:59:47.750018 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.749602    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"]
Jul 15 19:59:47.754017 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.753562    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"]
Jul 15 19:59:47.754017 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.753673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.757747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.757270    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 19:59:47.757747 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.757554    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 19:59:47.758669 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.758640    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 15 19:59:47.766245 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.766156    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 19:59:47.779126 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.779098    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9szdc"]
Jul 15 19:59:47.781689 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.781127    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:47.781689 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.781354    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-6srt7"]
Jul 15 19:59:47.783272 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.783253    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-6srt7"
Jul 15 19:59:47.788706 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.787952    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 19:59:47.788706 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.788255    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6rkdq\""
Jul 15 19:59:47.788706 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.788473    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 19:59:47.794689 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.794658    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-k95fh\""
Jul 15 19:59:47.794913 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.794896    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 15 19:59:47.795184 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.795167    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 15 19:59:47.824448 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.824420    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9szdc"]
Jul 15 19:59:47.831269 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.831140    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"]
Jul 15 19:59:47.833345 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.833322    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"
Jul 15 19:59:47.834019 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.833948    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-89dc4f459-k4gtm"]
Jul 15 19:59:47.837128 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.837108    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 19:59:47.837230 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.837108    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6j86x\""
Jul 15 19:59:47.860942 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.860909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52a14153-48da-4f0d-8f04-f7ea004b250b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9szdc\" (UID: \"52a14153-48da-4f0d-8f04-f7ea004b250b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:47.861153 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.860956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52a14153-48da-4f0d-8f04-f7ea004b250b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9szdc\" (UID: \"52a14153-48da-4f0d-8f04-f7ea004b250b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:47.861153 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.861006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg8j7\" (UniqueName: \"kubernetes.io/projected/8e001282-41de-470d-b1ff-88e93afe680a-kube-api-access-pg8j7\") pod \"downloads-68c9cbb497-6srt7\" (UID: \"8e001282-41de-470d-b1ff-88e93afe680a\") " pod="openshift-console/downloads-68c9cbb497-6srt7"
Jul 15 19:59:47.861153 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.861078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jplhc\" (UniqueName: \"kubernetes.io/projected/47651026-9051-49cf-be1a-03d9ea636028-kube-api-access-jplhc\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.861153 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.861134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/47651026-9051-49cf-be1a-03d9ea636028-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.861408 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.861168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47651026-9051-49cf-be1a-03d9ea636028-tmp\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.864992 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.864853    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-6srt7"]
Jul 15 19:59:47.865898 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.865854    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"]
Jul 15 19:59:47.944366 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.943345    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5867879996-sjtv7"]
Jul 15 19:59:47.945566 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.945548    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:47.961752 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.961726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/47651026-9051-49cf-be1a-03d9ea636028-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.961884 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.961770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47651026-9051-49cf-be1a-03d9ea636028-tmp\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.961884 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.961815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b984a64f-db50-4da4-8200-9b8d334ee230-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ncpf6\" (UID: \"b984a64f-db50-4da4-8200-9b8d334ee230\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"
Jul 15 19:59:47.961884 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.961878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52a14153-48da-4f0d-8f04-f7ea004b250b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9szdc\" (UID: \"52a14153-48da-4f0d-8f04-f7ea004b250b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:47.963625 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.961908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52a14153-48da-4f0d-8f04-f7ea004b250b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9szdc\" (UID: \"52a14153-48da-4f0d-8f04-f7ea004b250b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:47.963625 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.961932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg8j7\" (UniqueName: \"kubernetes.io/projected/8e001282-41de-470d-b1ff-88e93afe680a-kube-api-access-pg8j7\") pod \"downloads-68c9cbb497-6srt7\" (UID: \"8e001282-41de-470d-b1ff-88e93afe680a\") " pod="openshift-console/downloads-68c9cbb497-6srt7"
Jul 15 19:59:47.963625 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.961991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jplhc\" (UniqueName: \"kubernetes.io/projected/47651026-9051-49cf-be1a-03d9ea636028-kube-api-access-jplhc\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.964923 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.964307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52a14153-48da-4f0d-8f04-f7ea004b250b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-9szdc\" (UID: \"52a14153-48da-4f0d-8f04-f7ea004b250b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:47.964923 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.964859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47651026-9051-49cf-be1a-03d9ea636028-tmp\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.966501 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.966457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/47651026-9051-49cf-be1a-03d9ea636028-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:47.967509 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.967470    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52a14153-48da-4f0d-8f04-f7ea004b250b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-9szdc\" (UID: \"52a14153-48da-4f0d-8f04-f7ea004b250b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:47.985270 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:47.985247    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5867879996-sjtv7"]
Jul 15 19:59:48.003577 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.003513    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg8j7\" (UniqueName: \"kubernetes.io/projected/8e001282-41de-470d-b1ff-88e93afe680a-kube-api-access-pg8j7\") pod \"downloads-68c9cbb497-6srt7\" (UID: \"8e001282-41de-470d-b1ff-88e93afe680a\") " pod="openshift-console/downloads-68c9cbb497-6srt7"
Jul 15 19:59:48.018264 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.018130    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f5cc76fb-fxknp"]
Jul 15 19:59:48.024993 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.024884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jplhc\" (UniqueName: \"kubernetes.io/projected/47651026-9051-49cf-be1a-03d9ea636028-kube-api-access-jplhc\") pod \"klusterlet-addon-workmgr-7ff46477dd-h52pz\" (UID: \"47651026-9051-49cf-be1a-03d9ea636028\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:48.025394 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.025092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.032225 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.032173    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 19:59:48.032332 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.032235    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 19:59:48.032523 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.032501    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 19:59:48.033838 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.033431    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 19:59:48.033838 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.033693    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 19:59:48.034011 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.033941    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kj6kq\""
Jul 15 19:59:48.055101 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.055077    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5cc76fb-fxknp"]
Jul 15 19:59:48.063125 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8604be60-3471-4606-8e13-46f3470b44df-installation-pull-secrets\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.063243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-serving-cert\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.063243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8604be60-3471-4606-8e13-46f3470b44df-ca-trust-extracted\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.063243 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-service-ca\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.063408 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-bound-sa-token\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.063408 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b984a64f-db50-4da4-8200-9b8d334ee230-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ncpf6\" (UID: \"b984a64f-db50-4da4-8200-9b8d334ee230\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"
Jul 15 19:59:48.063408 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l2xz\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-kube-api-access-5l2xz\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.063554 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vct29\" (UniqueName: \"kubernetes.io/projected/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-kube-api-access-vct29\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.063554 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-oauth-serving-cert\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.063554 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8604be60-3471-4606-8e13-46f3470b44df-registry-certificates\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.063554 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063545    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-registry-tls\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.063791 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-config\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.063791 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-oauth-config\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.063791 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8604be60-3471-4606-8e13-46f3470b44df-trusted-ca\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.063791 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.063711    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8604be60-3471-4606-8e13-46f3470b44df-image-registry-private-configuration\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.065734 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.065696    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:48.066102 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.066083    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b984a64f-db50-4da4-8200-9b8d334ee230-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ncpf6\" (UID: \"b984a64f-db50-4da4-8200-9b8d334ee230\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"
Jul 15 19:59:48.096317 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.096294    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc"
Jul 15 19:59:48.102108 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.102087    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-6srt7"
Jul 15 19:59:48.144099 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.144074    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"
Jul 15 19:59:48.165045 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-service-ca\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.165150 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-bound-sa-token\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.165150 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l2xz\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-kube-api-access-5l2xz\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.165150 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vct29\" (UniqueName: \"kubernetes.io/projected/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-kube-api-access-vct29\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.165267 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-oauth-serving-cert\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.165428 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8604be60-3471-4606-8e13-46f3470b44df-registry-certificates\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.165486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165443    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-registry-tls\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.165486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-config\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.165568 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-oauth-config\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.165568 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165545    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8604be60-3471-4606-8e13-46f3470b44df-trusted-ca\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.165664 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8604be60-3471-4606-8e13-46f3470b44df-image-registry-private-configuration\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.165664 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8604be60-3471-4606-8e13-46f3470b44df-installation-pull-secrets\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.166177 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165784    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-oauth-serving-cert\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.166177 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-serving-cert\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.166177 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8604be60-3471-4606-8e13-46f3470b44df-ca-trust-extracted\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.166393 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.166194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8604be60-3471-4606-8e13-46f3470b44df-ca-trust-extracted\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.166393 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.166220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-config\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.166393 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.165785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-service-ca\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.166393 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.166359    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8604be60-3471-4606-8e13-46f3470b44df-registry-certificates\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.166588 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.166472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8604be60-3471-4606-8e13-46f3470b44df-trusted-ca\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.168498 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.168473    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-registry-tls\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.168794 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.168778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-serving-cert\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.168854 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.168777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8604be60-3471-4606-8e13-46f3470b44df-image-registry-private-configuration\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.168854 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.168821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-oauth-config\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.169099 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.169082    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8604be60-3471-4606-8e13-46f3470b44df-installation-pull-secrets\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.186486 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.186437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-bound-sa-token\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.188170 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.188143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vct29\" (UniqueName: \"kubernetes.io/projected/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-kube-api-access-vct29\") pod \"console-6f5cc76fb-fxknp\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") " pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:48.190745 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.190725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l2xz\" (UniqueName: \"kubernetes.io/projected/8604be60-3471-4606-8e13-46f3470b44df-kube-api-access-5l2xz\") pod \"image-registry-5867879996-sjtv7\" (UID: \"8604be60-3471-4606-8e13-46f3470b44df\") " pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.258605 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.258537    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:48.340854 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:48.340814    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:50.771475 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:50.771443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5cc76fb-fxknp"]
Jul 15 19:59:50.773947 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:50.773921    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc291bb49_1ff9_4d50_9c6e_fdfe2dd706a3.slice/crio-fefdbde65331f1d92bfba4a72a1114fd78f2c0d152d0f6b030d53e6fc62d66b8 WatchSource:0}: Error finding container fefdbde65331f1d92bfba4a72a1114fd78f2c0d152d0f6b030d53e6fc62d66b8: Status 404 returned error can't find the container with id fefdbde65331f1d92bfba4a72a1114fd78f2c0d152d0f6b030d53e6fc62d66b8
Jul 15 19:59:50.917919 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:50.917851    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-9szdc"]
Jul 15 19:59:50.920356 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:50.920329    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a14153_48da_4f0d_8f04_f7ea004b250b.slice/crio-b18f3308ff351a7fe6bde466012b1649700ed130b6c4f2ae973f3837778252fb WatchSource:0}: Error finding container b18f3308ff351a7fe6bde466012b1649700ed130b6c4f2ae973f3837778252fb: Status 404 returned error can't find the container with id b18f3308ff351a7fe6bde466012b1649700ed130b6c4f2ae973f3837778252fb
Jul 15 19:59:50.923397 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:50.923333    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"]
Jul 15 19:59:50.924714 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:50.924660    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5867879996-sjtv7"]
Jul 15 19:59:50.925778 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:50.925743    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"]
Jul 15 19:59:50.926770 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:50.926749    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-6srt7"]
Jul 15 19:59:50.928649 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:50.928622    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47651026_9051_49cf_be1a_03d9ea636028.slice/crio-8bdf6c7433bee99ba3a8c9a63a89ac40270567e63a922d6ae5006ea8c30b8406 WatchSource:0}: Error finding container 8bdf6c7433bee99ba3a8c9a63a89ac40270567e63a922d6ae5006ea8c30b8406: Status 404 returned error can't find the container with id 8bdf6c7433bee99ba3a8c9a63a89ac40270567e63a922d6ae5006ea8c30b8406
Jul 15 19:59:50.929178 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:50.929159    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb984a64f_db50_4da4_8200_9b8d334ee230.slice/crio-b4ec8a83ebbe45033eafcd659128b0baaf9d072c47ac51f58cd4eed255c8f976 WatchSource:0}: Error finding container b4ec8a83ebbe45033eafcd659128b0baaf9d072c47ac51f58cd4eed255c8f976: Status 404 returned error can't find the container with id b4ec8a83ebbe45033eafcd659128b0baaf9d072c47ac51f58cd4eed255c8f976
Jul 15 19:59:50.930009 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:50.929903    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8604be60_3471_4606_8e13_46f3470b44df.slice/crio-ff69795bfaddd2841d5b503f8dbe8c7e371beee24112dd73204d103f175fc937 WatchSource:0}: Error finding container ff69795bfaddd2841d5b503f8dbe8c7e371beee24112dd73204d103f175fc937: Status 404 returned error can't find the container with id ff69795bfaddd2841d5b503f8dbe8c7e371beee24112dd73204d103f175fc937
Jul 15 19:59:50.931011 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:50.930990    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e001282_41de_470d_b1ff_88e93afe680a.slice/crio-9dea4c2717beda4a29e6ecbe96f2ad6bbf2af32895f06b011a507570c81f2895 WatchSource:0}: Error finding container 9dea4c2717beda4a29e6ecbe96f2ad6bbf2af32895f06b011a507570c81f2895: Status 404 returned error can't find the container with id 9dea4c2717beda4a29e6ecbe96f2ad6bbf2af32895f06b011a507570c81f2895
Jul 15 19:59:51.374628 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.374593    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zh2mc" event={"ID":"41d3dc94-0044-4326-92be-d1550a0502b1","Type":"ContainerStarted","Data":"f4404de3db1358ca170ddf79c8fe649e3efd18d28e674cefa046d1640c5049d9"}
Jul 15 19:59:51.375726 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.375699    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc" event={"ID":"52a14153-48da-4f0d-8f04-f7ea004b250b","Type":"ContainerStarted","Data":"b18f3308ff351a7fe6bde466012b1649700ed130b6c4f2ae973f3837778252fb"}
Jul 15 19:59:51.376727 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.376705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz" event={"ID":"47651026-9051-49cf-be1a-03d9ea636028","Type":"ContainerStarted","Data":"8bdf6c7433bee99ba3a8c9a63a89ac40270567e63a922d6ae5006ea8c30b8406"}
Jul 15 19:59:51.378235 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.378213    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54fvs" event={"ID":"3bd9fdcd-774e-495b-888f-ddfe727138bf","Type":"ContainerStarted","Data":"a01cb596fd192c7c985e00177739cb721681a142e80294c2427d6e3540bb6f22"}
Jul 15 19:59:51.378325 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.378259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54fvs" event={"ID":"3bd9fdcd-774e-495b-888f-ddfe727138bf","Type":"ContainerStarted","Data":"5da3bb6085d528ef9f2d582588fca6e03221877b52d0677a9bd3cfe1b3dc571b"}
Jul 15 19:59:51.378372 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.378342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-54fvs"
Jul 15 19:59:51.379388 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.379369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cc76fb-fxknp" event={"ID":"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3","Type":"ContainerStarted","Data":"fefdbde65331f1d92bfba4a72a1114fd78f2c0d152d0f6b030d53e6fc62d66b8"}
Jul 15 19:59:51.380658 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.380639    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5867879996-sjtv7" event={"ID":"8604be60-3471-4606-8e13-46f3470b44df","Type":"ContainerStarted","Data":"b0d5a01d54d95433be2f5f2d19500ce52abf3a0666064e790a47feae6153dd18"}
Jul 15 19:59:51.380755 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.380669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5867879996-sjtv7" event={"ID":"8604be60-3471-4606-8e13-46f3470b44df","Type":"ContainerStarted","Data":"ff69795bfaddd2841d5b503f8dbe8c7e371beee24112dd73204d103f175fc937"}
Jul 15 19:59:51.380755 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.380729    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 19:59:51.381935 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.381917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ft87c" event={"ID":"afe79d63-635a-4f4d-9e66-d134b6ab4ba6","Type":"ContainerStarted","Data":"bd5c63a7f6052ff10d34c3a5120179733786eca471ed739c0bb71dcb3453421a"}
Jul 15 19:59:51.382042 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.381980    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 19:59:51.383013 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.382994    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-6srt7" event={"ID":"8e001282-41de-470d-b1ff-88e93afe680a","Type":"ContainerStarted","Data":"9dea4c2717beda4a29e6ecbe96f2ad6bbf2af32895f06b011a507570c81f2895"}
Jul 15 19:59:51.383982 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.383953    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6" event={"ID":"b984a64f-db50-4da4-8200-9b8d334ee230","Type":"ContainerStarted","Data":"b4ec8a83ebbe45033eafcd659128b0baaf9d072c47ac51f58cd4eed255c8f976"}
Jul 15 19:59:51.395807 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.395772    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zh2mc" podStartSLOduration=66.965362199 podStartE2EDuration="1m12.395759789s" totalImagesPullingTime="5.43039759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:51.395073049 +0000 UTC m=+72.843446146" watchObservedRunningTime="2026-07-15 19:59:51.395759789 +0000 UTC m=+72.844132863"
Jul 15 19:59:51.418033 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.417992    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5867879996-sjtv7" podStartSLOduration=4.417962307 podStartE2EDuration="4.417962307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:51.416839348 +0000 UTC m=+72.865212445" watchObservedRunningTime="2026-07-15 19:59:51.417962307 +0000 UTC m=+72.866335381"
Jul 15 19:59:51.438712 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.438675    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ft87c" podStartSLOduration=67.30562083 podStartE2EDuration="1m12.438652715s" totalImagesPullingTime="5.133031885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:51.437855282 +0000 UTC m=+72.886228378" watchObservedRunningTime="2026-07-15 19:59:51.438652715 +0000 UTC m=+72.887025789"
Jul 15 19:59:51.466018 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:51.465960    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-54fvs" podStartSLOduration=33.907161925 podStartE2EDuration="37.465946683s" totalImagesPullingTime="3.558784758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:51.464953144 +0000 UTC m=+72.913326241" watchObservedRunningTime="2026-07-15 19:59:51.465946683 +0000 UTC m=+72.914319780"
Jul 15 19:59:52.907140 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:52.906775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:52.912694 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:52.912264    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f3d2f159-5f80-4228-bedd-8d261a02e1f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vxzc\" (UID: \"f3d2f159-5f80-4228-bedd-8d261a02e1f4\") " pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:53.125856 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:53.125816    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v5glv\""
Jul 15 19:59:53.133659 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:53.133621    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2vxzc"
Jul 15 19:59:56.919381 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:56.918837    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2vxzc"]
Jul 15 19:59:56.922952 ip-10-0-130-52 kubenswrapper[2562]: W0715 19:59:56.922920    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d2f159_5f80_4228_bedd_8d261a02e1f4.slice/crio-cce1f6cc5e3f32d0e645d3c819e83f2c1fdbda29c97cc1fb76eb9d4bf0e468bd WatchSource:0}: Error finding container cce1f6cc5e3f32d0e645d3c819e83f2c1fdbda29c97cc1fb76eb9d4bf0e468bd: Status 404 returned error can't find the container with id cce1f6cc5e3f32d0e645d3c819e83f2c1fdbda29c97cc1fb76eb9d4bf0e468bd
Jul 15 19:59:57.419578 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.419535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc" event={"ID":"52a14153-48da-4f0d-8f04-f7ea004b250b","Type":"ContainerStarted","Data":"67f75cd9e45ab26e875117a29a59b03c25f4adffbd2938d19e881beeff5107cc"}
Jul 15 19:59:57.423225 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.423196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz" event={"ID":"47651026-9051-49cf-be1a-03d9ea636028","Type":"ContainerStarted","Data":"555491d7b48c374aed6ceaa3afd5812bcafab18e1e41fc3db68339c3128c22e1"}
Jul 15 19:59:57.426787 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.426760    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cc76fb-fxknp" event={"ID":"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3","Type":"ContainerStarted","Data":"1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa"}
Jul 15 19:59:57.431209 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.429033    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6" event={"ID":"b984a64f-db50-4da4-8200-9b8d334ee230","Type":"ContainerStarted","Data":"d5ba8e75af340f38d3f539af660d49c3ccb21acec56b3d40205e7447a032d867"}
Jul 15 19:59:57.431440 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.431387    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"
Jul 15 19:59:57.434331 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.434280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vxzc" event={"ID":"f3d2f159-5f80-4228-bedd-8d261a02e1f4","Type":"ContainerStarted","Data":"cce1f6cc5e3f32d0e645d3c819e83f2c1fdbda29c97cc1fb76eb9d4bf0e468bd"}
Jul 15 19:59:57.439547 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.439527    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6"
Jul 15 19:59:57.461460 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.460905    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-9szdc" podStartSLOduration=4.646769643 podStartE2EDuration="10.460885113s" totalImagesPullingTime="5.81411547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:57.439685149 +0000 UTC m=+78.888058248" watchObservedRunningTime="2026-07-15 19:59:57.460885113 +0000 UTC m=+78.909258209"
Jul 15 19:59:57.489997 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.486825    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ncpf6" podStartSLOduration=4.6815272 podStartE2EDuration="10.486758247s" totalImagesPullingTime="5.805231047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:57.46111237 +0000 UTC m=+78.909485479" watchObservedRunningTime="2026-07-15 19:59:57.486758247 +0000 UTC m=+78.935131344"
Jul 15 19:59:57.513356 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.511940    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f5cc76fb-fxknp" podStartSLOduration=4.541530252 podStartE2EDuration="10.511918246s" totalImagesPullingTime="5.970387994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:57.492755467 +0000 UTC m=+78.941128562" watchObservedRunningTime="2026-07-15 19:59:57.511918246 +0000 UTC m=+78.960291343"
Jul 15 19:59:57.513356 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:57.512130    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz" podStartSLOduration=4.670060918 podStartE2EDuration="10.512122212s" totalImagesPullingTime="5.842061294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:57.5115375 +0000 UTC m=+78.959910597" watchObservedRunningTime="2026-07-15 19:59:57.512122212 +0000 UTC m=+78.960495309"
Jul 15 19:59:58.077022 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.076984    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jpbvn"]
Jul 15 19:59:58.080256 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.080231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.083601 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.083577    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 15 19:59:58.083601 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.083597    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 19:59:58.083797 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.083778    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 15 19:59:58.084694 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.084671    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2xwqp\""
Jul 15 19:59:58.085129 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.084920    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 19:59:58.085129 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.084953    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 19:59:58.091497 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.091474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jpbvn"]
Jul 15 19:59:58.155232 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.155177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.155424 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.155254    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.155424 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.155298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8ml2\" (UniqueName: \"kubernetes.io/projected/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-kube-api-access-n8ml2\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.155539 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.155457    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.256468 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.256422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.256658 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.256510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.256658 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.256566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.256658 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.256608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8ml2\" (UniqueName: \"kubernetes.io/projected/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-kube-api-access-n8ml2\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.257032 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:58.257004    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 15 19:59:58.257123 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:58.257082    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls podName:126a7c91-08a6-4a2b-8da0-6ac52e50b17d nodeName:}" failed. No retries permitted until 2026-07-15 19:59:58.757064595 +0000 UTC m=+80.205437669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls") pod "prometheus-operator-64785b6675-jpbvn" (UID: "126a7c91-08a6-4a2b-8da0-6ac52e50b17d") : secret "prometheus-operator-tls" not found
Jul 15 19:59:58.257274 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.257250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.260220 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.260194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.266262 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.266228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8ml2\" (UniqueName: \"kubernetes.io/projected/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-kube-api-access-n8ml2\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.341285 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.341201    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:58.341285 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.341283    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:58.347333 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.347285    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:58.438455 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.438334    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:58.439084 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.439059    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff46477dd-h52pz"
Jul 15 19:59:58.443928 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.443908    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 19:59:58.761620 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:58.761523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:58.761838 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:58.761697    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 15 19:59:58.761838 ip-10-0-130-52 kubenswrapper[2562]: E0715 19:59:58.761777    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls podName:126a7c91-08a6-4a2b-8da0-6ac52e50b17d nodeName:}" failed. No retries permitted until 2026-07-15 19:59:59.761756232 +0000 UTC m=+81.210129311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls") pod "prometheus-operator-64785b6675-jpbvn" (UID: "126a7c91-08a6-4a2b-8da0-6ac52e50b17d") : secret "prometheus-operator-tls" not found
Jul 15 19:59:59.771895 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:59.771314    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:59.774444 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:59.774389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/126a7c91-08a6-4a2b-8da0-6ac52e50b17d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jpbvn\" (UID: \"126a7c91-08a6-4a2b-8da0-6ac52e50b17d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 19:59:59.893403 ip-10-0-130-52 kubenswrapper[2562]: I0715 19:59:59.893366    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn"
Jul 15 20:00:01.399584 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:01.399457    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-54fvs"
Jul 15 20:00:02.637155 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:02.637119    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jpbvn"]
Jul 15 20:00:03.462876 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:03.462830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vxzc" event={"ID":"f3d2f159-5f80-4228-bedd-8d261a02e1f4","Type":"ContainerStarted","Data":"ce4724c0f7431fc7775d33242616f7e17f5df02ba568edf61d2ce24c6fa54188"}
Jul 15 20:00:05.783962 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:05.783924    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5cc76fb-fxknp"]
Jul 15 20:00:09.373205 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:09.373171    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 20:00:09.661243 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:09.661168    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126a7c91_08a6_4a2b_8da0_6ac52e50b17d.slice/crio-34cb955f9e683781d7a678913f3d7267871bc47e1ad3cdce71895e81d3121089 WatchSource:0}: Error finding container 34cb955f9e683781d7a678913f3d7267871bc47e1ad3cdce71895e81d3121089: Status 404 returned error can't find the container with id 34cb955f9e683781d7a678913f3d7267871bc47e1ad3cdce71895e81d3121089
Jul 15 20:00:10.504704 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:10.504660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn" event={"ID":"126a7c91-08a6-4a2b-8da0-6ac52e50b17d","Type":"ContainerStarted","Data":"34cb955f9e683781d7a678913f3d7267871bc47e1ad3cdce71895e81d3121089"}
Jul 15 20:00:10.508063 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:10.508024    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-6srt7" event={"ID":"8e001282-41de-470d-b1ff-88e93afe680a","Type":"ContainerStarted","Data":"2f278e2d0ca444a9cf1926eca6ae77cf33aeb3d63829e788e3d13fc4665ddf17"}
Jul 15 20:00:10.508903 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:10.508877    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-6srt7"
Jul 15 20:00:10.523844 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:10.523772    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-6srt7"
Jul 15 20:00:10.529300 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:10.529241    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-6srt7" podStartSLOduration=4.489776621 podStartE2EDuration="23.529221528s" totalImagesPullingTime="19.039444907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:10.528587693 +0000 UTC m=+91.976960787" watchObservedRunningTime="2026-07-15 20:00:10.529221528 +0000 UTC m=+91.977594627"
Jul 15 20:00:12.402411 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:12.402303    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5867879996-sjtv7"
Jul 15 20:00:14.387829 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:14.387787    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm" podUID="11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" containerName="registry" containerID="cri-o://493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00" gracePeriod=30
Jul 15 20:00:14.881813 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:14.881775    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 20:00:15.002715 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002636    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-image-registry-private-configuration\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.002715 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002690    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-certificates\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.002949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002734    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqjnw\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-kube-api-access-kqjnw\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.002949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002770    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-trusted-ca\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.002949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002800    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.002949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002832    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-bound-sa-token\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.002949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002891    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-ca-trust-extracted\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.002949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.002933    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-installation-pull-secrets\") pod \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\" (UID: \"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e\") "
Jul 15 20:00:15.004072 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.004039    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:15.004667 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.004628    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:15.006874 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.006820    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:00:15.009480 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.009382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:00:15.009670 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.009571    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:00:15.009670 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.009645    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:00:15.009961 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.009929    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-kube-api-access-kqjnw" (OuterVolumeSpecName: "kube-api-access-kqjnw") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "kube-api-access-kqjnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:00:15.019723 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.019689    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" (UID: "11f8b0cf-010e-4a3d-8e2b-4e6e3add672e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:00:15.103873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103844    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqjnw\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-kube-api-access-kqjnw\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.103873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103873    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-trusted-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.103873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103883    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.104114 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103890    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-bound-sa-token\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.104114 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103898    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-ca-trust-extracted\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.104114 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103912    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-installation-pull-secrets\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.104114 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103927    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-image-registry-private-configuration\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.104114 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.103941    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e-registry-certificates\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:15.532093 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.532053    2562 generic.go:356] "Generic (PLEG): container finished" podID="11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" containerID="493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00" exitCode=0
Jul 15 20:00:15.532656 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.532154    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm" event={"ID":"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e","Type":"ContainerDied","Data":"493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00"}
Jul 15 20:00:15.532656 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.532205    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm" event={"ID":"11f8b0cf-010e-4a3d-8e2b-4e6e3add672e","Type":"ContainerDied","Data":"722c09b67da0eb3c0e22796ae113a290175fc4c59944041cedfdbc3409c2188e"}
Jul 15 20:00:15.532656 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.532235    2562 scope.go:117] "RemoveContainer" containerID="493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00"
Jul 15 20:00:15.532656 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.532409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89dc4f459-k4gtm"
Jul 15 20:00:15.535232 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.535155    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn" event={"ID":"126a7c91-08a6-4a2b-8da0-6ac52e50b17d","Type":"ContainerStarted","Data":"8b21f1a7d44629a1c28f5f6ec3c8a6aa9d983c4a80c3f65a8558cab7c7dee22c"}
Jul 15 20:00:15.535232 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.535206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn" event={"ID":"126a7c91-08a6-4a2b-8da0-6ac52e50b17d","Type":"ContainerStarted","Data":"f86447d1481cabe5973ea028eb7847a6e883420fb484756c850b13638539813e"}
Jul 15 20:00:15.537587 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.537563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vxzc" event={"ID":"f3d2f159-5f80-4228-bedd-8d261a02e1f4","Type":"ContainerStarted","Data":"a6957b3c7d16abd4c9e52fad42856ca9bc062e2773733e972d2add34665be15d"}
Jul 15 20:00:15.548942 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.548921    2562 scope.go:117] "RemoveContainer" containerID="493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00"
Jul 15 20:00:15.549315 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:00:15.549284    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00\": container with ID starting with 493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00 not found: ID does not exist" containerID="493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00"
Jul 15 20:00:15.549417 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.549320    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00"} err="failed to get container status \"493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00\": rpc error: code = NotFound desc = could not find container \"493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00\": container with ID starting with 493ab4bd5092f699c893206385497d2ac1c5124c9bba2f29c40c9fbba0cc5e00 not found: ID does not exist"
Jul 15 20:00:15.558260 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.558208    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-jpbvn" podStartSLOduration=12.494246111 podStartE2EDuration="17.558186977s" totalImagesPullingTime="5.063940866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:15.557423817 +0000 UTC m=+97.005796914" watchObservedRunningTime="2026-07-15 20:00:15.558186977 +0000 UTC m=+97.006560073"
Jul 15 20:00:15.588236 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.588202    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-89dc4f459-k4gtm"]
Jul 15 20:00:15.592144 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:15.592086    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-89dc4f459-k4gtm"]
Jul 15 20:00:17.078424 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.078389    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" path="/var/lib/kubelet/pods/11f8b0cf-010e-4a3d-8e2b-4e6e3add672e/volumes"
Jul 15 20:00:17.753187 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.753154    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-4p974"]
Jul 15 20:00:17.753553 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.753533    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" containerName="registry"
Jul 15 20:00:17.753739 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.753557    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" containerName="registry"
Jul 15 20:00:17.753739 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.753629    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="11f8b0cf-010e-4a3d-8e2b-4e6e3add672e" containerName="registry"
Jul 15 20:00:17.770428 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.770376    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d7m9l"]
Jul 15 20:00:17.770605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.770516    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:17.773467 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.773431    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 15 20:00:17.773636 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.773612    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qtrfj\""
Jul 15 20:00:17.776646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.776528    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 15 20:00:17.783348 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.782937    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 15 20:00:17.797011 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.796927    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-4p974"]
Jul 15 20:00:17.797235 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.797101    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.800893 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.800440    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 20:00:17.800893 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.800664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 20:00:17.802581 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.802557    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f5fs6\""
Jul 15 20:00:17.804305 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.804140    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 20:00:17.928508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-sys\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:17.928508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-root\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-accelerators-collector-config\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928488    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:17.928508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqxxt\" (UniqueName: \"kubernetes.io/projected/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-kube-api-access-gqxxt\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928525    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-wtmp\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928640    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-textfile\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928685    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-metrics-client-ca\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928742    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-tls\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:17.928954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:17.928812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xnkf\" (UniqueName: \"kubernetes.io/projected/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-api-access-7xnkf\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.030353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030262    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.030353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030318    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-textfile\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030373    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-metrics-client-ca\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-tls\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030458    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xnkf\" (UniqueName: \"kubernetes.io/projected/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-api-access-7xnkf\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-sys\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-root\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-accelerators-collector-config\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030632    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.030991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030653    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxxt\" (UniqueName: \"kubernetes.io/projected/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-kube-api-access-gqxxt\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-textfile\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030679    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-wtmp\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030828    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-wtmp\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-root\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.030991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.030869    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-sys\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.031752 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.031724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.031752 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.031739    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.032120 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.032096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.032208 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.032126    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-metrics-client-ca\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.033244 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.033202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-accelerators-collector-config\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.034344 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.034318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.034450 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.034368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.035099 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.035075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.037520 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.037479    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-node-exporter-tls\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.045022 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.044792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xnkf\" (UniqueName: \"kubernetes.io/projected/1dbe4d63-7e0b-4e0c-af84-2dd518d0b103-kube-api-access-7xnkf\") pod \"kube-state-metrics-785778ff74-4p974\" (UID: \"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.047659 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.047627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqxxt\" (UniqueName: \"kubernetes.io/projected/bb58ad33-fa45-4f75-9aee-1e8262cb1c35-kube-api-access-gqxxt\") pod \"node-exporter-d7m9l\" (UID: \"bb58ad33-fa45-4f75-9aee-1e8262cb1c35\") " pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.085656 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.085620    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974"
Jul 15 20:00:18.110214 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.110178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d7m9l"
Jul 15 20:00:18.155119 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:18.155087    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb58ad33_fa45_4f75_9aee_1e8262cb1c35.slice/crio-192f37bdae8d5b8464f016fe01bca87038ec2d290ab833dccb65a94ca22c7cbd WatchSource:0}: Error finding container 192f37bdae8d5b8464f016fe01bca87038ec2d290ab833dccb65a94ca22c7cbd: Status 404 returned error can't find the container with id 192f37bdae8d5b8464f016fe01bca87038ec2d290ab833dccb65a94ca22c7cbd
Jul 15 20:00:18.308275 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.308191    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-4p974"]
Jul 15 20:00:18.313166 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:18.312955    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dbe4d63_7e0b_4e0c_af84_2dd518d0b103.slice/crio-de07c0d59c2e00d1f0f76c35610da777ee90a868ccd28504a7edb9b979a1f345 WatchSource:0}: Error finding container de07c0d59c2e00d1f0f76c35610da777ee90a868ccd28504a7edb9b979a1f345: Status 404 returned error can't find the container with id de07c0d59c2e00d1f0f76c35610da777ee90a868ccd28504a7edb9b979a1f345
Jul 15 20:00:18.552908 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.552821    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d7m9l" event={"ID":"bb58ad33-fa45-4f75-9aee-1e8262cb1c35","Type":"ContainerStarted","Data":"192f37bdae8d5b8464f016fe01bca87038ec2d290ab833dccb65a94ca22c7cbd"}
Jul 15 20:00:18.554448 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:18.554414    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974" event={"ID":"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103","Type":"ContainerStarted","Data":"de07c0d59c2e00d1f0f76c35610da777ee90a868ccd28504a7edb9b979a1f345"}
Jul 15 20:00:20.666767 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.666685    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"]
Jul 15 20:00:20.701814 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.701778    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"]
Jul 15 20:00:20.701986 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.701898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.705484 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.705454    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 20:00:20.705628 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.705509    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 20:00:20.705628 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.705533    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rn4cq\""
Jul 15 20:00:20.705738 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.705685    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 20:00:20.706092 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.706073    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 20:00:20.706195 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.706122    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 20:00:20.706195 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.706142    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3hicaspl30o26\""
Jul 15 20:00:20.754564 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.754564 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-grpc-tls\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.754778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754623    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gdq5\" (UniqueName: \"kubernetes.io/projected/2ce6a864-c87e-4dbb-817c-dedc1c940431-kube-api-access-4gdq5\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.754778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6a864-c87e-4dbb-817c-dedc1c940431-metrics-client-ca\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.754778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-tls\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.754778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.754778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.754778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.754753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.855586 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gdq5\" (UniqueName: \"kubernetes.io/projected/2ce6a864-c87e-4dbb-817c-dedc1c940431-kube-api-access-4gdq5\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.855773 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6a864-c87e-4dbb-817c-dedc1c940431-metrics-client-ca\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.855773 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-tls\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.855905 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.855905 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.855905 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.856081 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855921    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.856081 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.855950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-grpc-tls\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.856420 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.856394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6a864-c87e-4dbb-817c-dedc1c940431-metrics-client-ca\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.858894 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.858868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.859026 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.858936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.859555 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.859533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-tls\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.859628 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.859604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-grpc-tls\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.860232 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.860210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.860368 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.860343    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ce6a864-c87e-4dbb-817c-dedc1c940431-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:20.864729 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:20.864706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gdq5\" (UniqueName: \"kubernetes.io/projected/2ce6a864-c87e-4dbb-817c-dedc1c940431-kube-api-access-4gdq5\") pod \"thanos-querier-66c9f7d96d-5nh7v\" (UID: \"2ce6a864-c87e-4dbb-817c-dedc1c940431\") " pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:21.013133 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:21.013095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:22.182134 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.182105    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78f57cd896-2j8tk"]
Jul 15 20:00:22.204547 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.204517    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78f57cd896-2j8tk"]
Jul 15 20:00:22.204708 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.204614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.211133 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.211113    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 20:00:22.212260 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.212242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 15 20:00:22.217337 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.217315    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 15 20:00:22.217422 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.217348    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1n4cp3dkmh1bi\""
Jul 15 20:00:22.217536 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.217523    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5kprb\""
Jul 15 20:00:22.217600 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.217573    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 15 20:00:22.267945 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.267919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-metrics-server-audit-profiles\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.268132 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.267963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.268132 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.268026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-client-ca-bundle\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.268132 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.268055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-secret-metrics-server-client-certs\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.268132 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.268104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc72s\" (UniqueName: \"kubernetes.io/projected/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-kube-api-access-sc72s\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.268287 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.268160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-audit-log\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.268287 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.268218    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-secret-metrics-server-tls\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.368881 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.368838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.368881 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.368874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-client-ca-bundle\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.369110 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.368895    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-secret-metrics-server-client-certs\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.369110 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.368920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc72s\" (UniqueName: \"kubernetes.io/projected/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-kube-api-access-sc72s\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.369110 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.368991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-audit-log\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.369110 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.369054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-secret-metrics-server-tls\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.369110 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.369097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-metrics-server-audit-profiles\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.369592 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.369556    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-audit-log\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.370000 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.369957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.370192 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.370175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-metrics-server-audit-profiles\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.371355 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.371334    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-secret-metrics-server-tls\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.371457 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.371434    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-client-ca-bundle\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.371559 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.371541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-secret-metrics-server-client-certs\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.398600 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.398578    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ft87c"
Jul 15 20:00:22.412781 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.412753    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc72s\" (UniqueName: \"kubernetes.io/projected/6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6-kube-api-access-sc72s\") pod \"metrics-server-78f57cd896-2j8tk\" (UID: \"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6\") " pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.473112 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.472059    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59f7445657-m5z4x"]
Jul 15 20:00:22.501732 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.500567    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59f7445657-m5z4x"]
Jul 15 20:00:22.501732 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.500692    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.506379 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:00:22.505618    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap"
Jul 15 20:00:22.506379 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:00:22.505692    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-59f7445657-m5z4x\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.513837 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.513812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:22.571219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.570931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-config\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.571219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.571011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-service-ca\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.571219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.571055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-oauth-serving-cert\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.571219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.571108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-serving-cert\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.571219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.571148    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-oauth-config\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.571219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.571181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.571219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.571207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2cpq\" (UniqueName: \"kubernetes.io/projected/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-kube-api-access-g2cpq\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.611507 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.611442    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"]
Jul 15 20:00:22.615910 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:22.615858    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce6a864_c87e_4dbb_817c_dedc1c940431.slice/crio-8182ada32c3722c4de594c36bdf1648424fc766e662133ab326b8d1d484de796 WatchSource:0}: Error finding container 8182ada32c3722c4de594c36bdf1648424fc766e662133ab326b8d1d484de796: Status 404 returned error can't find the container with id 8182ada32c3722c4de594c36bdf1648424fc766e662133ab326b8d1d484de796
Jul 15 20:00:22.673316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.672562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.673316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.672609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2cpq\" (UniqueName: \"kubernetes.io/projected/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-kube-api-access-g2cpq\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.673316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.672671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-config\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.673316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.672693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-service-ca\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.673316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.672728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-oauth-serving-cert\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.673316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.672778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-serving-cert\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.673316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.672819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-oauth-config\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.674942 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.674564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-oauth-serving-cert\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.674942 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.674634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-config\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.674942 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.674801    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-service-ca\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.676071 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.676046    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-oauth-config\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.677276 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.677253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-serving-cert\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.689675 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.689647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2cpq\" (UniqueName: \"kubernetes.io/projected/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-kube-api-access-g2cpq\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:22.739812 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:22.739787    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78f57cd896-2j8tk"]
Jul 15 20:00:22.755158 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:22.755128    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6aa1e8_4483_4e2b_acaa_e18b0340d3f6.slice/crio-15d599b5bd8f0a36bc5878c6907a558da8b7c2ae6a8ff2c0c36720c7735f7267 WatchSource:0}: Error finding container 15d599b5bd8f0a36bc5878c6907a558da8b7c2ae6a8ff2c0c36720c7735f7267: Status 404 returned error can't find the container with id 15d599b5bd8f0a36bc5878c6907a558da8b7c2ae6a8ff2c0c36720c7735f7267
Jul 15 20:00:23.579821 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.579783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974" event={"ID":"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103","Type":"ContainerStarted","Data":"ef03a04470fcfc3b45a88b0dd86e07f3d3e990d84fc293e2b2d258794f9881da"}
Jul 15 20:00:23.580440 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.579860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974" event={"ID":"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103","Type":"ContainerStarted","Data":"001eeac1bc3bd986394f1e0e7c8ec93bf875a61cbdeae6e14f0af5e278f07583"}
Jul 15 20:00:23.580440 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.579890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974" event={"ID":"1dbe4d63-7e0b-4e0c-af84-2dd518d0b103","Type":"ContainerStarted","Data":"9c799f15bb3dee8683b868e82d1ab580ec445794d35d02f81499ae53fa337290"}
Jul 15 20:00:23.582021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.581990    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vxzc" event={"ID":"f3d2f159-5f80-4228-bedd-8d261a02e1f4","Type":"ContainerStarted","Data":"5ce5be64d56d43cf8cc63204afec0a32a8bccdaf9f01e465bedc74f741e93e1b"}
Jul 15 20:00:23.583828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.583805    2562 generic.go:356] "Generic (PLEG): container finished" podID="bb58ad33-fa45-4f75-9aee-1e8262cb1c35" containerID="66b361de0a5a114c29706304a6aa5ae8ba23457ecfe48ad941012e6d1e8f4b85" exitCode=0
Jul 15 20:00:23.583985 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.583888    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d7m9l" event={"ID":"bb58ad33-fa45-4f75-9aee-1e8262cb1c35","Type":"ContainerDied","Data":"66b361de0a5a114c29706304a6aa5ae8ba23457ecfe48ad941012e6d1e8f4b85"}
Jul 15 20:00:23.585506 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.585483    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk" event={"ID":"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6","Type":"ContainerStarted","Data":"15d599b5bd8f0a36bc5878c6907a558da8b7c2ae6a8ff2c0c36720c7735f7267"}
Jul 15 20:00:23.587089 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.587066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" event={"ID":"2ce6a864-c87e-4dbb-817c-dedc1c940431","Type":"ContainerStarted","Data":"8182ada32c3722c4de594c36bdf1648424fc766e662133ab326b8d1d484de796"}
Jul 15 20:00:23.605025 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.604957    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-4p974" podStartSLOduration=2.473890302 podStartE2EDuration="6.60493707s" totalImagesPullingTime="4.131046768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:23.603057437 +0000 UTC m=+105.051430535" watchObservedRunningTime="2026-07-15 20:00:23.60493707 +0000 UTC m=+105.053310169"
Jul 15 20:00:23.647384 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.647337    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2vxzc" podStartSLOduration=38.317458305 podStartE2EDuration="1m3.6473201s" totalImagesPullingTime="25.329861795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:59:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:23.645250518 +0000 UTC m=+105.093623615" watchObservedRunningTime="2026-07-15 20:00:23.6473201 +0000 UTC m=+105.095693195"
Jul 15 20:00:23.675687 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:00:23.675664    2562 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: failed to sync configmap cache: timed out waiting for the condition
Jul 15 20:00:23.675818 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:00:23.675747    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle podName:b8ebc6b3-73c2-4bf9-b00a-9019be3891f9 nodeName:}" failed. No retries permitted until 2026-07-15 20:00:24.1757258 +0000 UTC m=+105.624098878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle") pod "console-59f7445657-m5z4x" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9") : failed to sync configmap cache: timed out waiting for the condition
Jul 15 20:00:23.979322 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:23.979289    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 20:00:24.068674 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.066783    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:00:24.094338 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.094309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.101256 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.101223    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 20:00:24.103522 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.102921    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 20:00:24.103522 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.103085    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tbmmf\""
Jul 15 20:00:24.103522 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.103184    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 20:00:24.105828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.104149    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 20:00:24.105828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.104202    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 20:00:24.105828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.104414    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 20:00:24.105828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.104447    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 20:00:24.105828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.104154    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 20:00:24.105828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.104667    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 20:00:24.105828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.104730    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 20:00:24.114996 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.110604    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7bkasv0t47f08\""
Jul 15 20:00:24.114996 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.112204    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:00:24.114996 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.112663    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 20:00:24.114996 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.112763    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 20:00:24.119533 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.119498    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 20:00:24.121584 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.121566    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 20:00:24.186793 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.186736    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187004 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.186807    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187004 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.186838    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187004 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.186870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187004 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.186943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187004 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.186991    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:24.187253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187218    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9gmj\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-kube-api-access-l9gmj\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187321    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-web-config\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config-out\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187840 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187840 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187566    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.187991 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.187954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle\") pod \"console-59f7445657-m5z4x\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") " pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:24.289177 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config-out\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289177 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289430 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289430 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289430 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289430 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289636 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289825 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.289825 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289927    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.289977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9gmj\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-kube-api-access-l9gmj\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290125    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-web-config\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.290386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.290233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.291788 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.291502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.296034 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.293389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.296034 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.293759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.296034 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.294358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.296034 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.294805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.296710 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.296684    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config-out\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.297175 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.297150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.298112 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.298077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.298308 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.298247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.299147 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.299121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.299250 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.299232    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.299316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.299225    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.299463 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.299442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.299838 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.299818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-web-config\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.300699 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.300657    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.300797 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.300746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.300933 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.300903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.308014 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.307945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9gmj\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-kube-api-access-l9gmj\") pod \"prometheus-k8s-0\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.314876 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.314850    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:24.419523 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.419391    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:24.598045 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.597590    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d7m9l" event={"ID":"bb58ad33-fa45-4f75-9aee-1e8262cb1c35","Type":"ContainerStarted","Data":"dc4fc5d684264ab0a09194180524e7f3bde9c3dd1624680ab2a17e64404a19c0"}
Jul 15 20:00:24.598045 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.597654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d7m9l" event={"ID":"bb58ad33-fa45-4f75-9aee-1e8262cb1c35","Type":"ContainerStarted","Data":"99d3e79cb052562770b98c4e70bdda0b8d841fef6f5040634de31ee8f19f2a61"}
Jul 15 20:00:24.620385 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.620319    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d7m9l" podStartSLOduration=3.237801177 podStartE2EDuration="7.62029563s" totalImagesPullingTime="4.267422298s" totalInitContainerRuntime="115.072155ms" isStatefulPod=true podCreationTimestamp="2026-07-15 20:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:24.618515574 +0000 UTC m=+106.066888665" watchObservedRunningTime="2026-07-15 20:00:24.62029563 +0000 UTC m=+106.068668727"
Jul 15 20:00:24.816280 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.816202    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59f7445657-m5z4x"]
Jul 15 20:00:24.827778 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:24.827747    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ebc6b3_73c2_4bf9_b00a_9019be3891f9.slice/crio-8231013ca291b3cecf0294a87dfc50cdee651ea446215a64ba95d816fdc269b1 WatchSource:0}: Error finding container 8231013ca291b3cecf0294a87dfc50cdee651ea446215a64ba95d816fdc269b1: Status 404 returned error can't find the container with id 8231013ca291b3cecf0294a87dfc50cdee651ea446215a64ba95d816fdc269b1
Jul 15 20:00:24.850633 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:24.850610    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:00:24.854734 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:24.854704    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df49bc3_80c3_4e9a_8e35_b0ad1ad0a1bd.slice/crio-dbd42b308f6ae7e5173faf664f6b474d37714db760d684d960a3dce9f7c9e5c4 WatchSource:0}: Error finding container dbd42b308f6ae7e5173faf664f6b474d37714db760d684d960a3dce9f7c9e5c4: Status 404 returned error can't find the container with id dbd42b308f6ae7e5173faf664f6b474d37714db760d684d960a3dce9f7c9e5c4
Jul 15 20:00:25.605353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:25.605310    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" event={"ID":"2ce6a864-c87e-4dbb-817c-dedc1c940431","Type":"ContainerStarted","Data":"1055e5f73c6ca03440f4d7d62fd485aa40914c7ab045aaa71cb2ee3869edf7cc"}
Jul 15 20:00:25.605877 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:25.605408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" event={"ID":"2ce6a864-c87e-4dbb-817c-dedc1c940431","Type":"ContainerStarted","Data":"a41901c5bd8aa5410ea057949bca1d80d195ad38316bfadb3cc49cb2d41f101e"}
Jul 15 20:00:25.605877 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:25.605451    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" event={"ID":"2ce6a864-c87e-4dbb-817c-dedc1c940431","Type":"ContainerStarted","Data":"ac1f9ce6fe5e8103293215b8da7a911333cccdc072f13b2e33dce83013ee35c7"}
Jul 15 20:00:25.607582 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:25.607552    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f7445657-m5z4x" event={"ID":"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9","Type":"ContainerStarted","Data":"7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9"}
Jul 15 20:00:25.607709 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:25.607605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f7445657-m5z4x" event={"ID":"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9","Type":"ContainerStarted","Data":"8231013ca291b3cecf0294a87dfc50cdee651ea446215a64ba95d816fdc269b1"}
Jul 15 20:00:25.609830 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:25.609684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerStarted","Data":"dbd42b308f6ae7e5173faf664f6b474d37714db760d684d960a3dce9f7c9e5c4"}
Jul 15 20:00:25.634614 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:25.634557    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59f7445657-m5z4x" podStartSLOduration=3.634535021 podStartE2EDuration="3.634535021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:25.633178224 +0000 UTC m=+107.081551324" watchObservedRunningTime="2026-07-15 20:00:25.634535021 +0000 UTC m=+107.082908118"
Jul 15 20:00:27.619358 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.619318    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk" event={"ID":"6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6","Type":"ContainerStarted","Data":"0bcf59d5a9792ece003549e4ff0a28e47ea091e19df342505a43aa59b947f8d2"}
Jul 15 20:00:27.620937 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.620910    2562 generic.go:356] "Generic (PLEG): container finished" podID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3" exitCode=0
Jul 15 20:00:27.621086 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.620993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"}
Jul 15 20:00:27.643525 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.643474    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk" podStartSLOduration=1.552897424 podStartE2EDuration="5.643457331s" totalImagesPullingTime="4.090559907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:27.642526206 +0000 UTC m=+109.090899302" watchObservedRunningTime="2026-07-15 20:00:27.643457331 +0000 UTC m=+109.091830426"
Jul 15 20:00:27.847583 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.847551    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59f7445657-m5z4x"]
Jul 15 20:00:27.921695 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.921601    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bc7cd5dcd-dfp46"]
Jul 15 20:00:27.957103 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.956403    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc7cd5dcd-dfp46"]
Jul 15 20:00:27.957103 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:27.956530    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.028190 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.028149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-oauth-serving-cert\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.028190 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.028188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-serving-cert\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.028402 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.028209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-service-ca\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.028402 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.028311    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-oauth-config\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.028402 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.028394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lrkm\" (UniqueName: \"kubernetes.io/projected/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-kube-api-access-2lrkm\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.028498 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.028422    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-config\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.028498 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.028453    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-trusted-ca-bundle\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.129306 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.129263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-serving-cert\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.129306 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.129303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-service-ca\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.129539 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.129332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-oauth-config\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.129539 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.129376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lrkm\" (UniqueName: \"kubernetes.io/projected/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-kube-api-access-2lrkm\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.129539 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.129407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-config\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.129539 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.129425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-trusted-ca-bundle\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.129539 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.129460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-oauth-serving-cert\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.130325 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.130299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-service-ca\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.130457 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.130372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-oauth-serving-cert\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.130457 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.130373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-config\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.130793 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.130766    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-trusted-ca-bundle\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.132021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.131991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-oauth-config\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.132248 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.132196    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-serving-cert\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.144469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.144442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lrkm\" (UniqueName: \"kubernetes.io/projected/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-kube-api-access-2lrkm\") pod \"console-6bc7cd5dcd-dfp46\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") " pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.266342 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.266311    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:28.446037 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.446002    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc7cd5dcd-dfp46"]
Jul 15 20:00:28.450812 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:00:28.450783    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4d5747_b37c_4029_9fb8_fa1037b00cb3.slice/crio-56410881f6aaeda815fa42c2d0bb4ebf972d9983ffb08e8ed77cb1ac94b0c45b WatchSource:0}: Error finding container 56410881f6aaeda815fa42c2d0bb4ebf972d9983ffb08e8ed77cb1ac94b0c45b: Status 404 returned error can't find the container with id 56410881f6aaeda815fa42c2d0bb4ebf972d9983ffb08e8ed77cb1ac94b0c45b
Jul 15 20:00:28.629985 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.629926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc7cd5dcd-dfp46" event={"ID":"4e4d5747-b37c-4029-9fb8-fa1037b00cb3","Type":"ContainerStarted","Data":"91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe"}
Jul 15 20:00:28.630566 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.630011    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc7cd5dcd-dfp46" event={"ID":"4e4d5747-b37c-4029-9fb8-fa1037b00cb3","Type":"ContainerStarted","Data":"56410881f6aaeda815fa42c2d0bb4ebf972d9983ffb08e8ed77cb1ac94b0c45b"}
Jul 15 20:00:28.654174 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:28.654123    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bc7cd5dcd-dfp46" podStartSLOduration=1.654104855 podStartE2EDuration="1.654104855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:28.653443895 +0000 UTC m=+110.101816988" watchObservedRunningTime="2026-07-15 20:00:28.654104855 +0000 UTC m=+110.102477951"
Jul 15 20:00:29.641361 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:29.641261    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" event={"ID":"2ce6a864-c87e-4dbb-817c-dedc1c940431","Type":"ContainerStarted","Data":"37702d20b067dec1f9c3433edf9311a8b4dcc0e35eca4d711faca0c2a4627027"}
Jul 15 20:00:29.641361 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:29.641346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" event={"ID":"2ce6a864-c87e-4dbb-817c-dedc1c940431","Type":"ContainerStarted","Data":"565149cb7cf0374f158e97fe561f817ece5b0e1ad6f10ea051649287a2d0eda1"}
Jul 15 20:00:29.641867 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:29.641395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" event={"ID":"2ce6a864-c87e-4dbb-817c-dedc1c940431","Type":"ContainerStarted","Data":"23bfebd71a1eaf27e98bb4b196ffca1e913302e65e087e050c94c4c4ac65b774"}
Jul 15 20:00:29.668727 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:29.668675    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v" podStartSLOduration=3.5347755579999998 podStartE2EDuration="9.66865804s" totalImagesPullingTime="6.133882482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:29.666662687 +0000 UTC m=+111.115035782" watchObservedRunningTime="2026-07-15 20:00:29.66865804 +0000 UTC m=+111.117031136"
Jul 15 20:00:30.647113 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:30.647079    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:30.653956 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:30.653926    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66c9f7d96d-5nh7v"
Jul 15 20:00:30.807567 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:30.807490    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f5cc76fb-fxknp" podUID="c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" containerName="console" containerID="cri-o://1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa" gracePeriod=15
Jul 15 20:00:31.529381 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.529357    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 20:00:31.577162 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577132    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-config\") pod \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") "
Jul 15 20:00:31.577315 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577234    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vct29\" (UniqueName: \"kubernetes.io/projected/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-kube-api-access-vct29\") pod \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") "
Jul 15 20:00:31.577315 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577283    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-oauth-config\") pod \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") "
Jul 15 20:00:31.577486 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577316    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-serving-cert\") pod \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") "
Jul 15 20:00:31.577486 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577346    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-oauth-serving-cert\") pod \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") "
Jul 15 20:00:31.577486 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577376    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-service-ca\") pod \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\" (UID: \"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3\") "
Jul 15 20:00:31.577629 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577572    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-config" (OuterVolumeSpecName: "console-config") pod "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" (UID: "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:31.577682 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577665    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:31.577770 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577738    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" (UID: "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:31.577861 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.577844    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-service-ca" (OuterVolumeSpecName: "service-ca") pod "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" (UID: "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:31.579704 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.579677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" (UID: "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:00:31.579905 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.579883    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" (UID: "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:00:31.579985 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.579890    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-kube-api-access-vct29" (OuterVolumeSpecName: "kube-api-access-vct29") pod "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" (UID: "c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3"). InnerVolumeSpecName "kube-api-access-vct29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:00:31.653294 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.653218    2562 generic.go:356] "Generic (PLEG): container finished" podID="c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" containerID="1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa" exitCode=0
Jul 15 20:00:31.653294 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.653274    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cc76fb-fxknp"
Jul 15 20:00:31.653782 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.653297    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cc76fb-fxknp" event={"ID":"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3","Type":"ContainerDied","Data":"1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa"}
Jul 15 20:00:31.653782 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.653345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cc76fb-fxknp" event={"ID":"c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3","Type":"ContainerDied","Data":"fefdbde65331f1d92bfba4a72a1114fd78f2c0d152d0f6b030d53e6fc62d66b8"}
Jul 15 20:00:31.653782 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.653372    2562 scope.go:117] "RemoveContainer" containerID="1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa"
Jul 15 20:00:31.669467 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.669443    2562 scope.go:117] "RemoveContainer" containerID="1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa"
Jul 15 20:00:31.669828 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:00:31.669803    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa\": container with ID starting with 1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa not found: ID does not exist" containerID="1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa"
Jul 15 20:00:31.669936 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.669837    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa"} err="failed to get container status \"1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa\": rpc error: code = NotFound desc = could not find container \"1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa\": container with ID starting with 1f14e79685438005659d35c8d8132d2bb9a91cc5e39f60589a58ce33f559e9aa not found: ID does not exist"
Jul 15 20:00:31.678469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.678362    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-oauth-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:31.678469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.678387    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-console-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:31.678469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.678397    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-oauth-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:31.678469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.678407    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:31.678469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.678415    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vct29\" (UniqueName: \"kubernetes.io/projected/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3-kube-api-access-vct29\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:31.679729 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.679706    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5cc76fb-fxknp"]
Jul 15 20:00:31.685542 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:31.685508    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f5cc76fb-fxknp"]
Jul 15 20:00:32.660732 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:32.660686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerStarted","Data":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"}
Jul 15 20:00:32.661230 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:32.660804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerStarted","Data":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"}
Jul 15 20:00:32.661230 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:32.660834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerStarted","Data":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"}
Jul 15 20:00:32.661230 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:32.660869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerStarted","Data":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"}
Jul 15 20:00:32.661230 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:32.660910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerStarted","Data":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"}
Jul 15 20:00:32.661230 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:32.660951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerStarted","Data":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"}
Jul 15 20:00:32.693886 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:32.693832    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.605649062 podStartE2EDuration="8.693814242s" totalImagesPullingTime="6.051726383s" totalInitContainerRuntime="36.438797ms" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:32.692131635 +0000 UTC m=+114.140504733" watchObservedRunningTime="2026-07-15 20:00:32.693814242 +0000 UTC m=+114.142187339"
Jul 15 20:00:33.078551 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:33.078513    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" path="/var/lib/kubelet/pods/c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3/volumes"
Jul 15 20:00:34.315383 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:34.315348    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:34.419904 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:34.419867    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:00:38.266614 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:38.266579    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:38.267110 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:38.266655    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:38.272427 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:38.272406    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:38.689446 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:38.689422    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:00:42.515042 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:42.515006    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:42.515608 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:42.515083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:00:52.867482 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:52.867433    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59f7445657-m5z4x" podUID="b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" containerName="console" containerID="cri-o://7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9" gracePeriod=15
Jul 15 20:00:53.649051 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.649025    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:53.754024 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.753916    2562 generic.go:356] "Generic (PLEG): container finished" podID="b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" containerID="7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9" exitCode=0
Jul 15 20:00:53.754024 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.753991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f7445657-m5z4x" event={"ID":"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9","Type":"ContainerDied","Data":"7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9"}
Jul 15 20:00:53.754024 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.754014    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f7445657-m5z4x"
Jul 15 20:00:53.754300 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.754066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f7445657-m5z4x" event={"ID":"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9","Type":"ContainerDied","Data":"8231013ca291b3cecf0294a87dfc50cdee651ea446215a64ba95d816fdc269b1"}
Jul 15 20:00:53.754300 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.754095    2562 scope.go:117] "RemoveContainer" containerID="7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9"
Jul 15 20:00:53.762379 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.762360    2562 scope.go:117] "RemoveContainer" containerID="7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9"
Jul 15 20:00:53.762635 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:00:53.762615    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9\": container with ID starting with 7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9 not found: ID does not exist" containerID="7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9"
Jul 15 20:00:53.762705 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.762646    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9"} err="failed to get container status \"7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9\": rpc error: code = NotFound desc = could not find container \"7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9\": container with ID starting with 7b35254ef3b2a22d5fd053a8717ac507fb4fdba2683f633322ab5f4d0ee577f9 not found: ID does not exist"
Jul 15 20:00:53.783071 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783042    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-oauth-serving-cert\") pod \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") "
Jul 15 20:00:53.783193 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783095    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle\") pod \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") "
Jul 15 20:00:53.783193 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783124    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-service-ca\") pod \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") "
Jul 15 20:00:53.783193 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783176    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-serving-cert\") pod \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") "
Jul 15 20:00:53.783408 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783218    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-oauth-config\") pod \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") "
Jul 15 20:00:53.783408 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783275    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-config\") pod \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") "
Jul 15 20:00:53.783408 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783335    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2cpq\" (UniqueName: \"kubernetes.io/projected/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-kube-api-access-g2cpq\") pod \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\" (UID: \"b8ebc6b3-73c2-4bf9-b00a-9019be3891f9\") "
Jul 15 20:00:53.783552 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783505    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:53.783613 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783569    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-service-ca" (OuterVolumeSpecName: "service-ca") pod "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:53.783764 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783716    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-oauth-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:53.783764 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783740    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:53.783934 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783778    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-config" (OuterVolumeSpecName: "console-config") pod "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:53.783934 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.783820    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:00:53.785556 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.785536    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:00:53.785643 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.785613    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-kube-api-access-g2cpq" (OuterVolumeSpecName: "kube-api-access-g2cpq") pod "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9"). InnerVolumeSpecName "kube-api-access-g2cpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:00:53.786295 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.786274    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" (UID: "b8ebc6b3-73c2-4bf9-b00a-9019be3891f9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:00:53.884322 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.884290    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:53.884322 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.884317    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-oauth-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:53.884322 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.884326    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-console-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:53.884734 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.884335    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2cpq\" (UniqueName: \"kubernetes.io/projected/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-kube-api-access-g2cpq\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:53.884734 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:53.884344    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9-trusted-ca-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:00:54.075390 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:54.075365    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59f7445657-m5z4x"]
Jul 15 20:00:54.079901 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:54.079874    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59f7445657-m5z4x"]
Jul 15 20:00:55.080323 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:00:55.080295    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" path="/var/lib/kubelet/pods/b8ebc6b3-73c2-4bf9-b00a-9019be3891f9/volumes"
Jul 15 20:01:02.520581 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:02.520553    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:01:02.524528 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:02.524505    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78f57cd896-2j8tk"
Jul 15 20:01:24.419634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:24.419599    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:24.440823 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:24.440798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:24.887628 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:24.887603    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:42.603500 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.603461    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:01:42.603948 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.603777    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="prometheus" containerID="cri-o://e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc" gracePeriod=600
Jul 15 20:01:42.603948 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.603820    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="thanos-sidecar" containerID="cri-o://766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f" gracePeriod=600
Jul 15 20:01:42.603948 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.603818    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-web" containerID="cri-o://4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216" gracePeriod=600
Jul 15 20:01:42.603948 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.603797    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be" gracePeriod=600
Jul 15 20:01:42.603948 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.603818    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy" containerID="cri-o://e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07" gracePeriod=600
Jul 15 20:01:42.603948 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.603837    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="config-reloader" containerID="cri-o://f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8" gracePeriod=600
Jul 15 20:01:42.851167 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.850836    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:42.906366 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906292    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906366 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906328    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9gmj\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-kube-api-access-l9gmj\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906366 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-1\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-2\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906407    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-thanos-prometheus-http-client-file\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906437    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906476    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-trusted-ca-bundle\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906517    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-kubelet-serving-ca-bundle\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906549    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-grpc-tls\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906590    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-serving-certs-ca-bundle\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906617    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-metrics-client-certs\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.906646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906643    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-metrics-client-ca\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906670    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-tls\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906700    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-web-config\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906718    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906746    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config-out\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906783    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906814    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-tls-assets\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906859    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-db\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906884    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-0\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906921    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-kube-rbac-proxy\") pod \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\" (UID: \"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd\") "
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.906997    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:42.907109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.907113    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:42.907790 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.907218    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:42.907790 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.907240    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:42.907790 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.907240    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:42.907790 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.907255    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:42.907790 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.907484    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:42.908151 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.907796    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:42.910793 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.910748    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config" (OuterVolumeSpecName: "config") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.910921 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.910819    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.911277 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.911250    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config-out" (OuterVolumeSpecName: "config-out") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:01:42.911778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.911648    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:01:42.911778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.911730    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.911778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.911752    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:01:42.912233 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.912193    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.912422 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.912398    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.912504 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.912422    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:01:42.912504 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.912437    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-kube-api-access-l9gmj" (OuterVolumeSpecName: "kube-api-access-l9gmj") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "kube-api-access-l9gmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:01:42.912792 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.912768    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.913197 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.913176    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.913887 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.913862    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.924747 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.924720    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-web-config" (OuterVolumeSpecName: "web-config") pod "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" (UID: "7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:01:42.943508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943481    2562 generic.go:356] "Generic (PLEG): container finished" podID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be" exitCode=0
Jul 15 20:01:42.943508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943504    2562 generic.go:356] "Generic (PLEG): container finished" podID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07" exitCode=0
Jul 15 20:01:42.943508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943512    2562 generic.go:356] "Generic (PLEG): container finished" podID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216" exitCode=0
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943517    2562 generic.go:356] "Generic (PLEG): container finished" podID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f" exitCode=0
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943523    2562 generic.go:356] "Generic (PLEG): container finished" podID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8" exitCode=0
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943528    2562 generic.go:356] "Generic (PLEG): container finished" podID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc" exitCode=0
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943557    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"}
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943585    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"}
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"}
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943706    2562 scope.go:117] "RemoveContainer" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"}
Jul 15 20:01:42.943776 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"}
Jul 15 20:01:42.944180 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943814    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"}
Jul 15 20:01:42.944180 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.943854    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd","Type":"ContainerDied","Data":"dbd42b308f6ae7e5173faf664f6b474d37714db760d684d960a3dce9f7c9e5c4"}
Jul 15 20:01:42.951672 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.951652    2562 scope.go:117] "RemoveContainer" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:42.960127 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.960106    2562 scope.go:117] "RemoveContainer" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:42.968648 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.968614    2562 scope.go:117] "RemoveContainer" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:42.973982 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.973945    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:01:42.976995 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.976963    2562 scope.go:117] "RemoveContainer" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:42.978365 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.978342    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:01:42.983833 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.983817    2562 scope.go:117] "RemoveContainer" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:42.991727 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.991710    2562 scope.go:117] "RemoveContainer" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:42.997942 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.997929    2562 scope.go:117] "RemoveContainer" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:42.998203 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:01:42.998180    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": container with ID starting with 3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be not found: ID does not exist" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:42.998255 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998216    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"} err="failed to get container status \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": rpc error: code = NotFound desc = could not find container \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": container with ID starting with 3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be not found: ID does not exist"
Jul 15 20:01:42.998255 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998234    2562 scope.go:117] "RemoveContainer" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:42.998494 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:01:42.998478    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": container with ID starting with e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07 not found: ID does not exist" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:42.998531 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998501    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"} err="failed to get container status \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": rpc error: code = NotFound desc = could not find container \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": container with ID starting with e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07 not found: ID does not exist"
Jul 15 20:01:42.998531 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998515    2562 scope.go:117] "RemoveContainer" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:42.998709 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:01:42.998692    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": container with ID starting with 4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216 not found: ID does not exist" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:42.998745 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998715    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"} err="failed to get container status \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": rpc error: code = NotFound desc = could not find container \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": container with ID starting with 4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216 not found: ID does not exist"
Jul 15 20:01:42.998745 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998728    2562 scope.go:117] "RemoveContainer" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:42.998902 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:01:42.998887    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": container with ID starting with 766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f not found: ID does not exist" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:42.998945 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998907    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"} err="failed to get container status \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": rpc error: code = NotFound desc = could not find container \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": container with ID starting with 766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f not found: ID does not exist"
Jul 15 20:01:42.998945 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.998927    2562 scope.go:117] "RemoveContainer" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:42.999194 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:01:42.999178    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": container with ID starting with f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8 not found: ID does not exist" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:42.999241 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999196    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"} err="failed to get container status \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": rpc error: code = NotFound desc = could not find container \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": container with ID starting with f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8 not found: ID does not exist"
Jul 15 20:01:42.999241 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999211    2562 scope.go:117] "RemoveContainer" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:42.999409 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:01:42.999392    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": container with ID starting with e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc not found: ID does not exist" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:42.999448 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999416    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"} err="failed to get container status \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": rpc error: code = NotFound desc = could not find container \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": container with ID starting with e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc not found: ID does not exist"
Jul 15 20:01:42.999448 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999432    2562 scope.go:117] "RemoveContainer" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:42.999629 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:01:42.999610    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": container with ID starting with f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3 not found: ID does not exist" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:42.999675 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999631    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"} err="failed to get container status \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": rpc error: code = NotFound desc = could not find container \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": container with ID starting with f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3 not found: ID does not exist"
Jul 15 20:01:42.999675 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999641    2562 scope.go:117] "RemoveContainer" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:42.999857 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999842    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"} err="failed to get container status \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": rpc error: code = NotFound desc = could not find container \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": container with ID starting with 3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be not found: ID does not exist"
Jul 15 20:01:42.999899 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:42.999857    2562 scope.go:117] "RemoveContainer" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:43.000073 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000055    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"} err="failed to get container status \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": rpc error: code = NotFound desc = could not find container \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": container with ID starting with e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07 not found: ID does not exist"
Jul 15 20:01:43.000073 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000072    2562 scope.go:117] "RemoveContainer" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:43.000271 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000256    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"} err="failed to get container status \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": rpc error: code = NotFound desc = could not find container \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": container with ID starting with 4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216 not found: ID does not exist"
Jul 15 20:01:43.000325 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000271    2562 scope.go:117] "RemoveContainer" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:43.000475 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000460    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"} err="failed to get container status \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": rpc error: code = NotFound desc = could not find container \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": container with ID starting with 766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f not found: ID does not exist"
Jul 15 20:01:43.000514 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000476    2562 scope.go:117] "RemoveContainer" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:43.000702 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000683    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"} err="failed to get container status \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": rpc error: code = NotFound desc = could not find container \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": container with ID starting with f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8 not found: ID does not exist"
Jul 15 20:01:43.000745 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000703    2562 scope.go:117] "RemoveContainer" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:43.000918 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000900    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"} err="failed to get container status \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": rpc error: code = NotFound desc = could not find container \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": container with ID starting with e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc not found: ID does not exist"
Jul 15 20:01:43.000995 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.000918    2562 scope.go:117] "RemoveContainer" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:43.001164 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001146    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"} err="failed to get container status \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": rpc error: code = NotFound desc = could not find container \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": container with ID starting with f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3 not found: ID does not exist"
Jul 15 20:01:43.001230 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001167    2562 scope.go:117] "RemoveContainer" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:43.001371 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001351    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"} err="failed to get container status \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": rpc error: code = NotFound desc = could not find container \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": container with ID starting with 3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be not found: ID does not exist"
Jul 15 20:01:43.001421 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001372    2562 scope.go:117] "RemoveContainer" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:43.001555 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001541    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"} err="failed to get container status \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": rpc error: code = NotFound desc = could not find container \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": container with ID starting with e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07 not found: ID does not exist"
Jul 15 20:01:43.001595 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001555    2562 scope.go:117] "RemoveContainer" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:43.001768 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001751    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"} err="failed to get container status \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": rpc error: code = NotFound desc = could not find container \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": container with ID starting with 4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216 not found: ID does not exist"
Jul 15 20:01:43.001810 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001769    2562 scope.go:117] "RemoveContainer" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:43.001954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001934    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"} err="failed to get container status \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": rpc error: code = NotFound desc = could not find container \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": container with ID starting with 766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f not found: ID does not exist"
Jul 15 20:01:43.002021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.001955    2562 scope.go:117] "RemoveContainer" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:43.002155 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002137    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"} err="failed to get container status \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": rpc error: code = NotFound desc = could not find container \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": container with ID starting with f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8 not found: ID does not exist"
Jul 15 20:01:43.002194 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002156    2562 scope.go:117] "RemoveContainer" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:43.002365 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002345    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"} err="failed to get container status \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": rpc error: code = NotFound desc = could not find container \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": container with ID starting with e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc not found: ID does not exist"
Jul 15 20:01:43.002403 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002366    2562 scope.go:117] "RemoveContainer" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:43.002561 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002545    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"} err="failed to get container status \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": rpc error: code = NotFound desc = could not find container \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": container with ID starting with f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3 not found: ID does not exist"
Jul 15 20:01:43.002617 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002562    2562 scope.go:117] "RemoveContainer" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:43.002783 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002766    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"} err="failed to get container status \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": rpc error: code = NotFound desc = could not find container \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": container with ID starting with 3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be not found: ID does not exist"
Jul 15 20:01:43.002826 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002783    2562 scope.go:117] "RemoveContainer" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:43.003002 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.002962    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"} err="failed to get container status \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": rpc error: code = NotFound desc = could not find container \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": container with ID starting with e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07 not found: ID does not exist"
Jul 15 20:01:43.003037 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003002    2562 scope.go:117] "RemoveContainer" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:43.003197 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003181    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"} err="failed to get container status \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": rpc error: code = NotFound desc = could not find container \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": container with ID starting with 4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216 not found: ID does not exist"
Jul 15 20:01:43.003243 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003198    2562 scope.go:117] "RemoveContainer" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:43.003401 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003385    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"} err="failed to get container status \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": rpc error: code = NotFound desc = could not find container \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": container with ID starting with 766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f not found: ID does not exist"
Jul 15 20:01:43.003441 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003401    2562 scope.go:117] "RemoveContainer" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:43.003602 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003584    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"} err="failed to get container status \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": rpc error: code = NotFound desc = could not find container \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": container with ID starting with f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8 not found: ID does not exist"
Jul 15 20:01:43.003642 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003603    2562 scope.go:117] "RemoveContainer" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:43.003810 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003793    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"} err="failed to get container status \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": rpc error: code = NotFound desc = could not find container \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": container with ID starting with e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc not found: ID does not exist"
Jul 15 20:01:43.003875 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.003812    2562 scope.go:117] "RemoveContainer" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:43.004020 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004004    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"} err="failed to get container status \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": rpc error: code = NotFound desc = could not find container \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": container with ID starting with f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3 not found: ID does not exist"
Jul 15 20:01:43.004089 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004022    2562 scope.go:117] "RemoveContainer" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:43.004205 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004188    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"} err="failed to get container status \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": rpc error: code = NotFound desc = could not find container \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": container with ID starting with 3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be not found: ID does not exist"
Jul 15 20:01:43.004244 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004206    2562 scope.go:117] "RemoveContainer" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:43.004423 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004406    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"} err="failed to get container status \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": rpc error: code = NotFound desc = could not find container \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": container with ID starting with e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07 not found: ID does not exist"
Jul 15 20:01:43.004467 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004423    2562 scope.go:117] "RemoveContainer" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:43.004612 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004596    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"} err="failed to get container status \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": rpc error: code = NotFound desc = could not find container \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": container with ID starting with 4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216 not found: ID does not exist"
Jul 15 20:01:43.004653 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004612    2562 scope.go:117] "RemoveContainer" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:43.004799 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004781    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"} err="failed to get container status \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": rpc error: code = NotFound desc = could not find container \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": container with ID starting with 766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f not found: ID does not exist"
Jul 15 20:01:43.004864 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.004800    2562 scope.go:117] "RemoveContainer" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:43.005022 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005004    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"} err="failed to get container status \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": rpc error: code = NotFound desc = could not find container \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": container with ID starting with f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8 not found: ID does not exist"
Jul 15 20:01:43.005071 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005023    2562 scope.go:117] "RemoveContainer" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:43.005233 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005216    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"} err="failed to get container status \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": rpc error: code = NotFound desc = could not find container \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": container with ID starting with e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc not found: ID does not exist"
Jul 15 20:01:43.005281 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005233    2562 scope.go:117] "RemoveContainer" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:43.005451 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005432    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"} err="failed to get container status \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": rpc error: code = NotFound desc = could not find container \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": container with ID starting with f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3 not found: ID does not exist"
Jul 15 20:01:43.005501 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005453    2562 scope.go:117] "RemoveContainer" containerID="3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"
Jul 15 20:01:43.005651 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005635    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be"} err="failed to get container status \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": rpc error: code = NotFound desc = could not find container \"3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be\": container with ID starting with 3a5a1de8ffa7f6456e9d3ffa63fdc35e7ee51add2fa7590bfd46b673218af4be not found: ID does not exist"
Jul 15 20:01:43.005691 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005651    2562 scope.go:117] "RemoveContainer" containerID="e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"
Jul 15 20:01:43.005849 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005833    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07"} err="failed to get container status \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": rpc error: code = NotFound desc = could not find container \"e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07\": container with ID starting with e2291b664182fb9d8a8d19764b3d35f97814001e63466cab82972c563497fa07 not found: ID does not exist"
Jul 15 20:01:43.005886 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.005850    2562 scope.go:117] "RemoveContainer" containerID="4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"
Jul 15 20:01:43.006052 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006033    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216"} err="failed to get container status \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": rpc error: code = NotFound desc = could not find container \"4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216\": container with ID starting with 4804df808e890367708326c7aea1026adfc78a286e902f9f1fefe9f800640216 not found: ID does not exist"
Jul 15 20:01:43.006103 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006053    2562 scope.go:117] "RemoveContainer" containerID="766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"
Jul 15 20:01:43.006240 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006224    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f"} err="failed to get container status \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": rpc error: code = NotFound desc = could not find container \"766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f\": container with ID starting with 766f2f007dd19f4a02044fbbd24cab4bc2123c63be29774b2c6866930c93027f not found: ID does not exist"
Jul 15 20:01:43.006282 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006240    2562 scope.go:117] "RemoveContainer" containerID="f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"
Jul 15 20:01:43.006443 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006426    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8"} err="failed to get container status \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": rpc error: code = NotFound desc = could not find container \"f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8\": container with ID starting with f8a614a04311caafa98e3f845b9efc745067a3f67764ac6f09e44f31866a4fd8 not found: ID does not exist"
Jul 15 20:01:43.006507 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006445    2562 scope.go:117] "RemoveContainer" containerID="e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"
Jul 15 20:01:43.006641 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006626    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc"} err="failed to get container status \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": rpc error: code = NotFound desc = could not find container \"e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc\": container with ID starting with e6cc412aee6117db93de3bc86eadd2f2c21199cbe977a58309c363623df5d6dc not found: ID does not exist"
Jul 15 20:01:43.006689 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006642    2562 scope.go:117] "RemoveContainer" containerID="f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"
Jul 15 20:01:43.006816 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.006799    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3"} err="failed to get container status \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": rpc error: code = NotFound desc = could not find container \"f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3\": container with ID starting with f4a0a83634b7f6011820d7dbb2fd5586157cc99739fa773f6c07e8f514482fd3 not found: ID does not exist"
Jul 15 20:01:43.008192 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008173    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008192 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008193    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008203    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008215    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-grpc-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008228    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008236    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-metrics-client-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008246    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-configmap-metrics-client-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008255    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008263    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-web-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008270    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config-out\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008278    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008287    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-tls-assets\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008300    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-db\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008308    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008316    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-secret-kube-rbac-proxy\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008323    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.008332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.008330    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9gmj\" (UniqueName: \"kubernetes.io/projected/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd-kube-api-access-l9gmj\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036373    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036807    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="init-config-reloader"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036824    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="init-config-reloader"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036837    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-thanos"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036846    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-thanos"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036861    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036870    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036882    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="thanos-sidecar"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036890    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="thanos-sidecar"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036913    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="prometheus"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036921    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="prometheus"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036932    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" containerName="console"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036940    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" containerName="console"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036951    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-web"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036959    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-web"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036988    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="config-reloader"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.036996    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="config-reloader"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037007    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" containerName="console"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037015    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" containerName="console"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037085    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-web"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037094    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c291bb49-1ff9-4d50-9c6e-fdfe2dd706a3" containerName="console"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037108    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="config-reloader"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037115    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy-thanos"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037123    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ebc6b3-73c2-4bf9-b00a-9019be3891f9" containerName="console"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037130    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="prometheus"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037139    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="kube-rbac-proxy"
Jul 15 20:01:43.037512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.037147    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" containerName="thanos-sidecar"
Jul 15 20:01:43.041554 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.041536    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.058115 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.058096    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 20:01:43.058227 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.058099    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 20:01:43.058835 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.058652    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 20:01:43.058835 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.058654    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tbmmf\""
Jul 15 20:01:43.059324 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.059305    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 20:01:43.063431 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.059738    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 20:01:43.063431 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.059951    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 20:01:43.063431 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.062591    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7bkasv0t47f08\""
Jul 15 20:01:43.063431 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.062834    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 20:01:43.063743 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.063713    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 20:01:43.063876 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.063854    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 20:01:43.064660 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.064641    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 20:01:43.077734 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.077711    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd" path="/var/lib/kubelet/pods/7df49bc3-80c3-4e9a-8e35-b0ad1ad0a1bd/volumes"
Jul 15 20:01:43.085979 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.085954    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 20:01:43.086071 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.085996    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 20:01:43.086615 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.086578    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 20:01:43.088448 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.088423    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 20:01:43.109118 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109092    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5770e7f4-305b-48b5-86ae-5361439494a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109199 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109199 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109199 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109199 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8g9z\" (UniqueName: \"kubernetes.io/projected/5770e7f4-305b-48b5-86ae-5361439494a5-kube-api-access-b8g9z\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109216    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5770e7f4-305b-48b5-86ae-5361439494a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-config\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109326    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109530 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109530 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109530 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109425    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109530 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109679 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109679 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109679 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109783 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109678    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109783 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109708    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.109783 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.109732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.119679 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.119654    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:01:43.210266 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210230    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210266 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210288    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210306    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8g9z\" (UniqueName: \"kubernetes.io/projected/5770e7f4-305b-48b5-86ae-5361439494a5-kube-api-access-b8g9z\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5770e7f4-305b-48b5-86ae-5361439494a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-config\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210410    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.210474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.210679    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211222 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.211140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211662 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.211227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211662 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.211546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211837 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.211812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.211984 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.211951    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.212134 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.212110    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.212326 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.212307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.212443 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.212424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.212523 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.212502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5770e7f4-305b-48b5-86ae-5361439494a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.213363 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.213313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5770e7f4-305b-48b5-86ae-5361439494a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.213760 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.213717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.214609 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.214557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-config\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.215117 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.215080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.215270 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.215248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.215404 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.215381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.215810 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.215792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.215869 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.215839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5770e7f4-305b-48b5-86ae-5361439494a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.215869 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.215841    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.215942 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.215879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5770e7f4-305b-48b5-86ae-5361439494a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.216545 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.216528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.216643 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.216627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5770e7f4-305b-48b5-86ae-5361439494a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.221085 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.221064    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8g9z\" (UniqueName: \"kubernetes.io/projected/5770e7f4-305b-48b5-86ae-5361439494a5-kube-api-access-b8g9z\") pod \"prometheus-k8s-0\" (UID: \"5770e7f4-305b-48b5-86ae-5361439494a5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.350386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.350356    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:43.497300 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.497212    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 20:01:43.500550 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:01:43.500522    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5770e7f4_305b_48b5_86ae_5361439494a5.slice/crio-203b802b940f1978d688543236c0a2f725208b7083de28cd2c1755d9bbfe5e97 WatchSource:0}: Error finding container 203b802b940f1978d688543236c0a2f725208b7083de28cd2c1755d9bbfe5e97: Status 404 returned error can't find the container with id 203b802b940f1978d688543236c0a2f725208b7083de28cd2c1755d9bbfe5e97
Jul 15 20:01:43.949046 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.949014    2562 generic.go:356] "Generic (PLEG): container finished" podID="5770e7f4-305b-48b5-86ae-5361439494a5" containerID="1f68cd2464eb25f7ab815be65f1b5f4025411a8abc1528d9ca224ab0ef76478e" exitCode=0
Jul 15 20:01:43.949485 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.949103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerDied","Data":"1f68cd2464eb25f7ab815be65f1b5f4025411a8abc1528d9ca224ab0ef76478e"}
Jul 15 20:01:43.949485 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:43.949190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerStarted","Data":"203b802b940f1978d688543236c0a2f725208b7083de28cd2c1755d9bbfe5e97"}
Jul 15 20:01:44.957337 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:44.957304    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerStarted","Data":"2471fadd27e03ab1bc74b3d462887babc6609a4da1aa9643a18b58a325b1fe47"}
Jul 15 20:01:44.957716 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:44.957397    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerStarted","Data":"828e61bf1d304b49886c8c5d217618a43547cf4c5285005d577abf676e4d688f"}
Jul 15 20:01:44.957716 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:44.957425    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerStarted","Data":"793154c3c184de819e36c03c47533518307a6fd9282b883567dd778488d18d75"}
Jul 15 20:01:44.957716 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:44.957448    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerStarted","Data":"ba17c4c1f30075f2f660578844864e7c2d6ee88748c7bb9b85f169702443f451"}
Jul 15 20:01:44.957716 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:44.957471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerStarted","Data":"07dc61e6e2bc5daaa0fad4c9f8c3feeac07b227c39060631ac4f207e15d5e71c"}
Jul 15 20:01:44.957716 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:44.957492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5770e7f4-305b-48b5-86ae-5361439494a5","Type":"ContainerStarted","Data":"6ee1337d1208ae47c6a41e0aaa517ea850dd0f16f534f6aaea35d231f233580f"}
Jul 15 20:01:45.002328 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:45.002272    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.983579595 podStartE2EDuration="3.002255369s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.675774ms" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:45.001021463 +0000 UTC m=+186.449394559" watchObservedRunningTime="2026-07-15 20:01:45.002255369 +0000 UTC m=+186.450628464"
Jul 15 20:01:48.351558 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:48.351515    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:01:50.263905 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:01:50.263873    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bc7cd5dcd-dfp46"]
Jul 15 20:02:15.283659 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.283594    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bc7cd5dcd-dfp46" podUID="4e4d5747-b37c-4029-9fb8-fa1037b00cb3" containerName="console" containerID="cri-o://91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe" gracePeriod=15
Jul 15 20:02:15.537512 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.537453    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:02:15.702656 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.702626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-service-ca\") pod \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") "
Jul 15 20:02:15.702829 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.702685    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-serving-cert\") pod \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") "
Jul 15 20:02:15.702829 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.702713    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-config\") pod \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") "
Jul 15 20:02:15.702829 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.702745    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-oauth-config\") pod \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") "
Jul 15 20:02:15.702829 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.702820    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lrkm\" (UniqueName: \"kubernetes.io/projected/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-kube-api-access-2lrkm\") pod \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") "
Jul 15 20:02:15.703052 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.702847    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-trusted-ca-bundle\") pod \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") "
Jul 15 20:02:15.703052 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703029    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-oauth-serving-cert\") pod \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\" (UID: \"4e4d5747-b37c-4029-9fb8-fa1037b00cb3\") "
Jul 15 20:02:15.703267 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703106    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-service-ca" (OuterVolumeSpecName: "service-ca") pod "4e4d5747-b37c-4029-9fb8-fa1037b00cb3" (UID: "4e4d5747-b37c-4029-9fb8-fa1037b00cb3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:02:15.703267 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703213    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-config" (OuterVolumeSpecName: "console-config") pod "4e4d5747-b37c-4029-9fb8-fa1037b00cb3" (UID: "4e4d5747-b37c-4029-9fb8-fa1037b00cb3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:02:15.703382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703267    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4e4d5747-b37c-4029-9fb8-fa1037b00cb3" (UID: "4e4d5747-b37c-4029-9fb8-fa1037b00cb3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:02:15.703382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703343    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-trusted-ca-bundle\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:02:15.703382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703364    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:02:15.703382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703378    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:02:15.703699 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.703659    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4e4d5747-b37c-4029-9fb8-fa1037b00cb3" (UID: "4e4d5747-b37c-4029-9fb8-fa1037b00cb3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:02:15.704958 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.704931    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4e4d5747-b37c-4029-9fb8-fa1037b00cb3" (UID: "4e4d5747-b37c-4029-9fb8-fa1037b00cb3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:02:15.705243 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.705214    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-kube-api-access-2lrkm" (OuterVolumeSpecName: "kube-api-access-2lrkm") pod "4e4d5747-b37c-4029-9fb8-fa1037b00cb3" (UID: "4e4d5747-b37c-4029-9fb8-fa1037b00cb3"). InnerVolumeSpecName "kube-api-access-2lrkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:02:15.705346 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.705328    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4e4d5747-b37c-4029-9fb8-fa1037b00cb3" (UID: "4e4d5747-b37c-4029-9fb8-fa1037b00cb3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:02:15.804029 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.803922    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-oauth-config\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:02:15.804029 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.803950    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lrkm\" (UniqueName: \"kubernetes.io/projected/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-kube-api-access-2lrkm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:02:15.804029 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.803963    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-oauth-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:02:15.804029 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:15.803994    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e4d5747-b37c-4029-9fb8-fa1037b00cb3-console-serving-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:02:16.089759 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.089671    2562 generic.go:356] "Generic (PLEG): container finished" podID="4e4d5747-b37c-4029-9fb8-fa1037b00cb3" containerID="91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe" exitCode=0
Jul 15 20:02:16.089759 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.089714    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc7cd5dcd-dfp46"
Jul 15 20:02:16.089929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.089758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc7cd5dcd-dfp46" event={"ID":"4e4d5747-b37c-4029-9fb8-fa1037b00cb3","Type":"ContainerDied","Data":"91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe"}
Jul 15 20:02:16.089929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.089805    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc7cd5dcd-dfp46" event={"ID":"4e4d5747-b37c-4029-9fb8-fa1037b00cb3","Type":"ContainerDied","Data":"56410881f6aaeda815fa42c2d0bb4ebf972d9983ffb08e8ed77cb1ac94b0c45b"}
Jul 15 20:02:16.089929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.089830    2562 scope.go:117] "RemoveContainer" containerID="91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe"
Jul 15 20:02:16.098686 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.098661    2562 scope.go:117] "RemoveContainer" containerID="91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe"
Jul 15 20:02:16.098938 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:02:16.098921    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe\": container with ID starting with 91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe not found: ID does not exist" containerID="91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe"
Jul 15 20:02:16.099008 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.098945    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe"} err="failed to get container status \"91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe\": rpc error: code = NotFound desc = could not find container \"91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe\": container with ID starting with 91f4d97f7ea9a16d144ff1fde22746267e769d2fab97102a79e028a0dc3799fe not found: ID does not exist"
Jul 15 20:02:16.117283 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.117251    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bc7cd5dcd-dfp46"]
Jul 15 20:02:16.123252 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:16.123229    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bc7cd5dcd-dfp46"]
Jul 15 20:02:17.078531 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:17.078498    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e4d5747-b37c-4029-9fb8-fa1037b00cb3" path="/var/lib/kubelet/pods/4e4d5747-b37c-4029-9fb8-fa1037b00cb3/volumes"
Jul 15 20:02:43.350811 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:43.350773    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:02:43.366183 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:43.366157    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:02:44.213111 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:02:44.213083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 20:03:38.952423 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:03:38.952395    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 20:05:07.451483 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.451454    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"]
Jul 15 20:05:07.452140 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.451786    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e4d5747-b37c-4029-9fb8-fa1037b00cb3" containerName="console"
Jul 15 20:05:07.452140 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.451796    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e4d5747-b37c-4029-9fb8-fa1037b00cb3" containerName="console"
Jul 15 20:05:07.452140 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.451842    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e4d5747-b37c-4029-9fb8-fa1037b00cb3" containerName="console"
Jul 15 20:05:07.453861 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.453845    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.459458 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.459436    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b8zwh\""
Jul 15 20:05:07.459600 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.459582    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 20:05:07.459679 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.459658    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 20:05:07.459742 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.459664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 20:05:07.460259 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.460088    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 20:05:07.481174 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.481146    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"]
Jul 15 20:05:07.513873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.513840    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xqbk\" (UniqueName: \"kubernetes.io/projected/8bd82a0c-ed6b-4558-8d04-d93485554634-kube-api-access-9xqbk\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.514058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.513884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bd82a0c-ed6b-4558-8d04-d93485554634-tmp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.514058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.513942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bd82a0c-ed6b-4558-8d04-d93485554634-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.514058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.514006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bd82a0c-ed6b-4558-8d04-d93485554634-manifests\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.514058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.514034    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd82a0c-ed6b-4558-8d04-d93485554634-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.615116 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.615074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xqbk\" (UniqueName: \"kubernetes.io/projected/8bd82a0c-ed6b-4558-8d04-d93485554634-kube-api-access-9xqbk\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.615313 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.615130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bd82a0c-ed6b-4558-8d04-d93485554634-tmp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.615313 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.615178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bd82a0c-ed6b-4558-8d04-d93485554634-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.615313 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.615234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bd82a0c-ed6b-4558-8d04-d93485554634-manifests\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.615313 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.615272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd82a0c-ed6b-4558-8d04-d93485554634-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.615652 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.615624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8bd82a0c-ed6b-4558-8d04-d93485554634-manifests\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.617535 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.617506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bd82a0c-ed6b-4558-8d04-d93485554634-tmp\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.617618 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.617584    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8bd82a0c-ed6b-4558-8d04-d93485554634-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.617657 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.617635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd82a0c-ed6b-4558-8d04-d93485554634-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.641279 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.641249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xqbk\" (UniqueName: \"kubernetes.io/projected/8bd82a0c-ed6b-4558-8d04-d93485554634-kube-api-access-9xqbk\") pod \"opendatahub-operator-controller-manager-56f6d85d69-84g4h\" (UID: \"8bd82a0c-ed6b-4558-8d04-d93485554634\") " pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.764029 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.763923    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:07.945585 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.945543    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"]
Jul 15 20:05:07.947848 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:05:07.947823    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd82a0c_ed6b_4558_8d04_d93485554634.slice/crio-82a97b7e587d3a322aabbb650d02966ed3c67ae440550dc1adf3dce7fa5836dd WatchSource:0}: Error finding container 82a97b7e587d3a322aabbb650d02966ed3c67ae440550dc1adf3dce7fa5836dd: Status 404 returned error can't find the container with id 82a97b7e587d3a322aabbb650d02966ed3c67ae440550dc1adf3dce7fa5836dd
Jul 15 20:05:07.949775 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:07.949757    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:05:08.747245 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:08.747209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h" event={"ID":"8bd82a0c-ed6b-4558-8d04-d93485554634","Type":"ContainerStarted","Data":"82a97b7e587d3a322aabbb650d02966ed3c67ae440550dc1adf3dce7fa5836dd"}
Jul 15 20:05:11.762478 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:11.762438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h" event={"ID":"8bd82a0c-ed6b-4558-8d04-d93485554634","Type":"ContainerStarted","Data":"3e31c1e7a4b2cb55e0034fe0581386e6868ff13f3f8235f606c0b403d6c695b1"}
Jul 15 20:05:12.768155 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:12.768116    2562 generic.go:356] "Generic (PLEG): container finished" podID="8bd82a0c-ed6b-4558-8d04-d93485554634" containerID="3e31c1e7a4b2cb55e0034fe0581386e6868ff13f3f8235f606c0b403d6c695b1" exitCode=0
Jul 15 20:05:12.768525 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:12.768205    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h" event={"ID":"8bd82a0c-ed6b-4558-8d04-d93485554634","Type":"ContainerDied","Data":"3e31c1e7a4b2cb55e0034fe0581386e6868ff13f3f8235f606c0b403d6c695b1"}
Jul 15 20:05:13.775323 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:13.775289    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h" event={"ID":"8bd82a0c-ed6b-4558-8d04-d93485554634","Type":"ContainerStarted","Data":"595717b4ccd8e67c792ab65cd6c45598f060dcc2292f9edfbf2f826abd64d33d"}
Jul 15 20:05:13.775702 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:13.775412    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:13.802305 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:13.802255    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h" podStartSLOduration=2.277869994 podStartE2EDuration="6.802240325s" totalImagesPullingTime="3.491924371s" totalInitContainerRuntime="1.03244596s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:13.799752635 +0000 UTC m=+395.248125731" watchObservedRunningTime="2026-07-15 20:05:13.802240325 +0000 UTC m=+395.250613421"
Jul 15 20:05:24.782767 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:24.782737    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56f6d85d69-84g4h"
Jul 15 20:05:25.933356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.933319    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"]
Jul 15 20:05:25.942841 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.942818    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:25.946118 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.946097    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 20:05:25.946459 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.946437    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2hxmd\""
Jul 15 20:05:25.946712 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.946690    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 20:05:25.947099 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.947077    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"]
Jul 15 20:05:25.947218 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.947126    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 20:05:25.947284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.947271    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 20:05:25.947815 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:25.947801    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 20:05:26.066055 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.066025    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-metrics-cert\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.066055 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.066061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-cert\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.066284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.066101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-manager-config\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.066284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.066152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl96c\" (UniqueName: \"kubernetes.io/projected/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-kube-api-access-bl96c\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.166657 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.166623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-metrics-cert\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.166818 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.166667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-cert\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.166818 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.166707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-manager-config\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.166818 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.166740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl96c\" (UniqueName: \"kubernetes.io/projected/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-kube-api-access-bl96c\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.167492 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.167461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-manager-config\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.169158 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.169134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-metrics-cert\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.169253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.169174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-cert\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.176451 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.176427    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl96c\" (UniqueName: \"kubernetes.io/projected/4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62-kube-api-access-bl96c\") pod \"lws-controller-manager-67d9958876-sdtzq\" (UID: \"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62\") " pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.253952 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.253863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:26.395061 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.395038    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"]
Jul 15 20:05:26.397749 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:05:26.397721    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af4574e_f8dc_4eb9_8b66_6f16d0fb9f62.slice/crio-c433a68f5470565388ff8880d58f8aa048d1de432314989aacd9749adfe6103b WatchSource:0}: Error finding container c433a68f5470565388ff8880d58f8aa048d1de432314989aacd9749adfe6103b: Status 404 returned error can't find the container with id c433a68f5470565388ff8880d58f8aa048d1de432314989aacd9749adfe6103b
Jul 15 20:05:26.828173 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:26.828140    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq" event={"ID":"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62","Type":"ContainerStarted","Data":"c433a68f5470565388ff8880d58f8aa048d1de432314989aacd9749adfe6103b"}
Jul 15 20:05:29.842768 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:29.842727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq" event={"ID":"4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62","Type":"ContainerStarted","Data":"983c7f3382030e517e4e77e51ff71e4f330c756df7473e83e15f79485f11a88c"}
Jul 15 20:05:29.843272 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:29.842800    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:29.939424 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:29.939359    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq" podStartSLOduration=2.472422413 podStartE2EDuration="4.939334917s" totalImagesPullingTime="2.466912504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:29.938432172 +0000 UTC m=+411.386805269" watchObservedRunningTime="2026-07-15 20:05:29.939334917 +0000 UTC m=+411.387708018"
Jul 15 20:05:30.340382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.340345    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"]
Jul 15 20:05:30.344945 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.344928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.348162 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.348142    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 20:05:30.349058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.349041    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 20:05:30.349156 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.349069    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 20:05:30.349156 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.349108    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 20:05:30.349156 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.349108    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ff5pc\""
Jul 15 20:05:30.375109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.375083    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"]
Jul 15 20:05:30.404339 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.404311    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-tls-certs\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.404510 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.404367    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-tmp\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.404510 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.404454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmktt\" (UniqueName: \"kubernetes.io/projected/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-kube-api-access-zmktt\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.505059 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.505026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmktt\" (UniqueName: \"kubernetes.io/projected/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-kube-api-access-zmktt\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.505197 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.505079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-tls-certs\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.505197 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.505127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-tmp\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.507515 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.507488    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-tmp\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.507722 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.507701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-tls-certs\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.533492 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.533457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmktt\" (UniqueName: \"kubernetes.io/projected/1f51d7bd-6452-4eb9-a03a-211be39ca5dc-kube-api-access-zmktt\") pod \"kube-auth-proxy-6dcf7fc97d-2dfbs\" (UID: \"1f51d7bd-6452-4eb9-a03a-211be39ca5dc\") " pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.654915 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.654838    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"
Jul 15 20:05:30.834774 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.834740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs"]
Jul 15 20:05:30.841079 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:05:30.841044    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f51d7bd_6452_4eb9_a03a_211be39ca5dc.slice/crio-dc88be9762cef3d2af0f2133d925fd5f052b5854fac818bde1014e5f937437e9 WatchSource:0}: Error finding container dc88be9762cef3d2af0f2133d925fd5f052b5854fac818bde1014e5f937437e9: Status 404 returned error can't find the container with id dc88be9762cef3d2af0f2133d925fd5f052b5854fac818bde1014e5f937437e9
Jul 15 20:05:30.848190 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:30.848161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs" event={"ID":"1f51d7bd-6452-4eb9-a03a-211be39ca5dc","Type":"ContainerStarted","Data":"dc88be9762cef3d2af0f2133d925fd5f052b5854fac818bde1014e5f937437e9"}
Jul 15 20:05:34.866728 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:34.866615    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs" event={"ID":"1f51d7bd-6452-4eb9-a03a-211be39ca5dc","Type":"ContainerStarted","Data":"83276c4db0e2ea154bd2f32d150c2f0fdc7c335d31704c4775ece2aeaaabedb2"}
Jul 15 20:05:40.850809 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:40.850778    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67d9958876-sdtzq"
Jul 15 20:05:40.904503 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:05:40.904453    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6dcf7fc97d-2dfbs" podStartSLOduration=7.208543314 podStartE2EDuration="10.904436774s" totalImagesPullingTime="3.69589346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:34.894036929 +0000 UTC m=+416.342410024" watchObservedRunningTime="2026-07-15 20:05:40.904436774 +0000 UTC m=+422.352810130"
Jul 15 20:07:12.554091 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.554055    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"]
Jul 15 20:07:12.556338 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.556320    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"
Jul 15 20:07:12.561181 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.561153    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 20:07:12.561519 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.561244    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 20:07:12.562206 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.562186    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 20:07:12.564547 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.564530    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vh6tb\""
Jul 15 20:07:12.576755 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.576724    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"]
Jul 15 20:07:12.708896 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.708852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5djv\" (UniqueName: \"kubernetes.io/projected/694bb4c4-eed2-4908-93a2-b7dd01a579ff-kube-api-access-x5djv\") pod \"dns-operator-controller-manager-864dd78c4c-5grkm\" (UID: \"694bb4c4-eed2-4908-93a2-b7dd01a579ff\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"
Jul 15 20:07:12.810421 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.810323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5djv\" (UniqueName: \"kubernetes.io/projected/694bb4c4-eed2-4908-93a2-b7dd01a579ff-kube-api-access-x5djv\") pod \"dns-operator-controller-manager-864dd78c4c-5grkm\" (UID: \"694bb4c4-eed2-4908-93a2-b7dd01a579ff\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"
Jul 15 20:07:12.819694 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.819669    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5djv\" (UniqueName: \"kubernetes.io/projected/694bb4c4-eed2-4908-93a2-b7dd01a579ff-kube-api-access-x5djv\") pod \"dns-operator-controller-manager-864dd78c4c-5grkm\" (UID: \"694bb4c4-eed2-4908-93a2-b7dd01a579ff\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"
Jul 15 20:07:12.866681 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:12.866631    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"
Jul 15 20:07:13.008165 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:13.008142    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"]
Jul 15 20:07:13.011074 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:07:13.011046    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694bb4c4_eed2_4908_93a2_b7dd01a579ff.slice/crio-4f7da7be3a776e6df0099eb28be900dfa1fecb1363ea575e68474212968edef4 WatchSource:0}: Error finding container 4f7da7be3a776e6df0099eb28be900dfa1fecb1363ea575e68474212968edef4: Status 404 returned error can't find the container with id 4f7da7be3a776e6df0099eb28be900dfa1fecb1363ea575e68474212968edef4
Jul 15 20:07:13.275605 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:13.275572    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm" event={"ID":"694bb4c4-eed2-4908-93a2-b7dd01a579ff","Type":"ContainerStarted","Data":"4f7da7be3a776e6df0099eb28be900dfa1fecb1363ea575e68474212968edef4"}
Jul 15 20:07:16.289166 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.289123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm" event={"ID":"694bb4c4-eed2-4908-93a2-b7dd01a579ff","Type":"ContainerStarted","Data":"1f9f3f3df85f7a5d84ec2e8f0127e5541e4fa0ccc49d235ec02c6975ffba4f3e"}
Jul 15 20:07:16.289573 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.289235    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"
Jul 15 20:07:16.315651 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.315600    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm" podStartSLOduration=1.525554278 podStartE2EDuration="4.31558498s" totalImagesPullingTime="2.790030702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:07:16.313199884 +0000 UTC m=+517.761572980" watchObservedRunningTime="2026-07-15 20:07:16.31558498 +0000 UTC m=+517.763958075"
Jul 15 20:07:16.491331 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.491284    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"]
Jul 15 20:07:16.493710 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.493691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"
Jul 15 20:07:16.496435 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.496416    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nwt8l\""
Jul 15 20:07:16.507136 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.507107    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"]
Jul 15 20:07:16.647143 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.647046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvmr9\" (UniqueName: \"kubernetes.io/projected/04e28bed-0d43-4518-8b33-a15c8ef6ec92-kube-api-access-dvmr9\") pod \"authorino-operator-75cc8d5cb7-pm8g5\" (UID: \"04e28bed-0d43-4518-8b33-a15c8ef6ec92\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"
Jul 15 20:07:16.748400 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.748358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvmr9\" (UniqueName: \"kubernetes.io/projected/04e28bed-0d43-4518-8b33-a15c8ef6ec92-kube-api-access-dvmr9\") pod \"authorino-operator-75cc8d5cb7-pm8g5\" (UID: \"04e28bed-0d43-4518-8b33-a15c8ef6ec92\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"
Jul 15 20:07:16.762323 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.762292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvmr9\" (UniqueName: \"kubernetes.io/projected/04e28bed-0d43-4518-8b33-a15c8ef6ec92-kube-api-access-dvmr9\") pod \"authorino-operator-75cc8d5cb7-pm8g5\" (UID: \"04e28bed-0d43-4518-8b33-a15c8ef6ec92\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"
Jul 15 20:07:16.803232 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.803198    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"
Jul 15 20:07:16.971312 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:16.971272    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"]
Jul 15 20:07:16.973915 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:07:16.973891    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e28bed_0d43_4518_8b33_a15c8ef6ec92.slice/crio-7e2739e2a285b513d61870a15d9bb378143925bd8edae65547145e3a198aa4d7 WatchSource:0}: Error finding container 7e2739e2a285b513d61870a15d9bb378143925bd8edae65547145e3a198aa4d7: Status 404 returned error can't find the container with id 7e2739e2a285b513d61870a15d9bb378143925bd8edae65547145e3a198aa4d7
Jul 15 20:07:17.294316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:17.294279    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5" event={"ID":"04e28bed-0d43-4518-8b33-a15c8ef6ec92","Type":"ContainerStarted","Data":"7e2739e2a285b513d61870a15d9bb378143925bd8edae65547145e3a198aa4d7"}
Jul 15 20:07:19.306385 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:19.306350    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5" event={"ID":"04e28bed-0d43-4518-8b33-a15c8ef6ec92","Type":"ContainerStarted","Data":"5cd3f91cdcd132003099511ec2063471d92479a9df9e4b36e5cca24ff71a550f"}
Jul 15 20:07:19.306857 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:19.306415    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"
Jul 15 20:07:19.380881 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:19.380778    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5" podStartSLOduration=1.281983221 podStartE2EDuration="3.380762515s" totalImagesPullingTime="2.098779294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:07:19.377188224 +0000 UTC m=+520.825561321" watchObservedRunningTime="2026-07-15 20:07:19.380762515 +0000 UTC m=+520.829135657"
Jul 15 20:07:27.296675 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:27.296640    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5grkm"
Jul 15 20:07:30.312113 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:07:30.312084    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pm8g5"
Jul 15 20:08:17.953158 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:17.953119    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-nrkhd"]
Jul 15 20:08:17.956428 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:17.956407    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nrkhd"
Jul 15 20:08:17.959858 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:17.959835    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mx56w\""
Jul 15 20:08:17.980696 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:17.980663    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nrkhd"]
Jul 15 20:08:18.001137 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:18.001097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cdsv\" (UniqueName: \"kubernetes.io/projected/a23cae02-1bb3-4f72-85a2-0d5fb9a475ff-kube-api-access-7cdsv\") pod \"authorino-58cd955675-nrkhd\" (UID: \"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff\") " pod="kuadrant-system/authorino-58cd955675-nrkhd"
Jul 15 20:08:18.102129 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:18.102096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cdsv\" (UniqueName: \"kubernetes.io/projected/a23cae02-1bb3-4f72-85a2-0d5fb9a475ff-kube-api-access-7cdsv\") pod \"authorino-58cd955675-nrkhd\" (UID: \"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff\") " pod="kuadrant-system/authorino-58cd955675-nrkhd"
Jul 15 20:08:18.113463 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:18.113437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cdsv\" (UniqueName: \"kubernetes.io/projected/a23cae02-1bb3-4f72-85a2-0d5fb9a475ff-kube-api-access-7cdsv\") pod \"authorino-58cd955675-nrkhd\" (UID: \"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff\") " pod="kuadrant-system/authorino-58cd955675-nrkhd"
Jul 15 20:08:18.285200 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:18.285111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nrkhd"
Jul 15 20:08:18.435752 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:18.435720    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nrkhd"]
Jul 15 20:08:18.439505 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:08:18.439471    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda23cae02_1bb3_4f72_85a2_0d5fb9a475ff.slice/crio-70bbd4544467d5a3c0f7dbe64769885af164bf9cbf5c02e26135331e7f1a387d WatchSource:0}: Error finding container 70bbd4544467d5a3c0f7dbe64769885af164bf9cbf5c02e26135331e7f1a387d: Status 404 returned error can't find the container with id 70bbd4544467d5a3c0f7dbe64769885af164bf9cbf5c02e26135331e7f1a387d
Jul 15 20:08:18.576955 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:18.576870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nrkhd" event={"ID":"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff","Type":"ContainerStarted","Data":"70bbd4544467d5a3c0f7dbe64769885af164bf9cbf5c02e26135331e7f1a387d"}
Jul 15 20:08:21.596263 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:21.596224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nrkhd" event={"ID":"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff","Type":"ContainerStarted","Data":"48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9"}
Jul 15 20:08:21.670116 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:21.670058    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-nrkhd" podStartSLOduration=1.62889905 podStartE2EDuration="4.670042801s" totalImagesPullingTime="3.041143751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:08:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:08:21.669771589 +0000 UTC m=+583.118144679" watchObservedRunningTime="2026-07-15 20:08:21.670042801 +0000 UTC m=+583.118415946"
Jul 15 20:08:23.385170 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:23.385138    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nrkhd"]
Jul 15 20:08:23.604807 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:23.604767    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-nrkhd" podUID="a23cae02-1bb3-4f72-85a2-0d5fb9a475ff" containerName="authorino" containerID="cri-o://48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9" gracePeriod=30
Jul 15 20:08:23.861349 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:23.861324    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nrkhd"
Jul 15 20:08:23.955881 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:23.955851    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cdsv\" (UniqueName: \"kubernetes.io/projected/a23cae02-1bb3-4f72-85a2-0d5fb9a475ff-kube-api-access-7cdsv\") pod \"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff\" (UID: \"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff\") "
Jul 15 20:08:23.958060 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:23.958030    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a23cae02-1bb3-4f72-85a2-0d5fb9a475ff-kube-api-access-7cdsv" (OuterVolumeSpecName: "kube-api-access-7cdsv") pod "a23cae02-1bb3-4f72-85a2-0d5fb9a475ff" (UID: "a23cae02-1bb3-4f72-85a2-0d5fb9a475ff"). InnerVolumeSpecName "kube-api-access-7cdsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:08:24.057314 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.057235    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cdsv\" (UniqueName: \"kubernetes.io/projected/a23cae02-1bb3-4f72-85a2-0d5fb9a475ff-kube-api-access-7cdsv\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:08:24.609802 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.609770    2562 generic.go:356] "Generic (PLEG): container finished" podID="a23cae02-1bb3-4f72-85a2-0d5fb9a475ff" containerID="48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9" exitCode=0
Jul 15 20:08:24.610284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.609813    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nrkhd"
Jul 15 20:08:24.610284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.609850    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nrkhd" event={"ID":"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff","Type":"ContainerDied","Data":"48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9"}
Jul 15 20:08:24.610284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.609893    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nrkhd" event={"ID":"a23cae02-1bb3-4f72-85a2-0d5fb9a475ff","Type":"ContainerDied","Data":"70bbd4544467d5a3c0f7dbe64769885af164bf9cbf5c02e26135331e7f1a387d"}
Jul 15 20:08:24.610284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.609909    2562 scope.go:117] "RemoveContainer" containerID="48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9"
Jul 15 20:08:24.619802 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.619777    2562 scope.go:117] "RemoveContainer" containerID="48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9"
Jul 15 20:08:24.620425 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:08:24.620404    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9\": container with ID starting with 48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9 not found: ID does not exist" containerID="48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9"
Jul 15 20:08:24.620590 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.620567    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9"} err="failed to get container status \"48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9\": rpc error: code = NotFound desc = could not find container \"48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9\": container with ID starting with 48be04539774eeb51bddef1a7c5f5e2f4d647c070fb8c2b3ba45a3a6630d7ac9 not found: ID does not exist"
Jul 15 20:08:24.635741 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.635700    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nrkhd"]
Jul 15 20:08:24.646443 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:24.646415    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-nrkhd"]
Jul 15 20:08:25.079316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:08:25.079284    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a23cae02-1bb3-4f72-85a2-0d5fb9a475ff" path="/var/lib/kubelet/pods/a23cae02-1bb3-4f72-85a2-0d5fb9a475ff/volumes"
Jul 15 20:09:08.653526 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.653434    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86d98d5447-c24vm"]
Jul 15 20:09:08.654136 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.653932    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23cae02-1bb3-4f72-85a2-0d5fb9a475ff" containerName="authorino"
Jul 15 20:09:08.654136 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.653952    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23cae02-1bb3-4f72-85a2-0d5fb9a475ff" containerName="authorino"
Jul 15 20:09:08.654136 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.654079    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a23cae02-1bb3-4f72-85a2-0d5fb9a475ff" containerName="authorino"
Jul 15 20:09:08.657193 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.657169    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.660195 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.660170    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 20:09:08.660195 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.660183    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b8fdfb54d-khn4s"]
Jul 15 20:09:08.661396 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.661375    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x9wjp\""
Jul 15 20:09:08.661498 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.661444    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 20:09:08.663310 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.663291    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:08.675141 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.675102    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d98d5447-c24vm"]
Jul 15 20:09:08.678440 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.678413    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b8fdfb54d-khn4s"]
Jul 15 20:09:08.763424 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.763394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/179466fe-25e0-416f-9c70-94c39dc85818-webhook-cert\") pod \"maas-controller-5b8fdfb54d-khn4s\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") " pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:08.763595 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.763447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c88ce048-965d-40f0-b7a2-535dc21878f2-webhook-cert\") pod \"maas-controller-86d98d5447-c24vm\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") " pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.763595 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.763555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcvgz\" (UniqueName: \"kubernetes.io/projected/c88ce048-965d-40f0-b7a2-535dc21878f2-kube-api-access-dcvgz\") pod \"maas-controller-86d98d5447-c24vm\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") " pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.763690 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.763606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td8b7\" (UniqueName: \"kubernetes.io/projected/179466fe-25e0-416f-9c70-94c39dc85818-kube-api-access-td8b7\") pod \"maas-controller-5b8fdfb54d-khn4s\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") " pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:08.864977 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.864941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcvgz\" (UniqueName: \"kubernetes.io/projected/c88ce048-965d-40f0-b7a2-535dc21878f2-kube-api-access-dcvgz\") pod \"maas-controller-86d98d5447-c24vm\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") " pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.865157 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.865039    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td8b7\" (UniqueName: \"kubernetes.io/projected/179466fe-25e0-416f-9c70-94c39dc85818-kube-api-access-td8b7\") pod \"maas-controller-5b8fdfb54d-khn4s\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") " pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:08.865157 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.865073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/179466fe-25e0-416f-9c70-94c39dc85818-webhook-cert\") pod \"maas-controller-5b8fdfb54d-khn4s\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") " pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:08.865157 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.865119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c88ce048-965d-40f0-b7a2-535dc21878f2-webhook-cert\") pod \"maas-controller-86d98d5447-c24vm\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") " pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.867625 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.867593    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c88ce048-965d-40f0-b7a2-535dc21878f2-webhook-cert\") pod \"maas-controller-86d98d5447-c24vm\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") " pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.867748 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.867730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/179466fe-25e0-416f-9c70-94c39dc85818-webhook-cert\") pod \"maas-controller-5b8fdfb54d-khn4s\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") " pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:08.882165 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.882139    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcvgz\" (UniqueName: \"kubernetes.io/projected/c88ce048-965d-40f0-b7a2-535dc21878f2-kube-api-access-dcvgz\") pod \"maas-controller-86d98d5447-c24vm\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") " pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.883756 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.883736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td8b7\" (UniqueName: \"kubernetes.io/projected/179466fe-25e0-416f-9c70-94c39dc85818-kube-api-access-td8b7\") pod \"maas-controller-5b8fdfb54d-khn4s\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") " pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:08.967388 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.967360    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:08.975510 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:08.975476    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:09.124140 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:09.124114    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d98d5447-c24vm"]
Jul 15 20:09:09.125508 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:09:09.125477    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc88ce048_965d_40f0_b7a2_535dc21878f2.slice/crio-5400546f3769a44c814d0b2d4874571008dd2e1221d6e3273238d39665d7615b WatchSource:0}: Error finding container 5400546f3769a44c814d0b2d4874571008dd2e1221d6e3273238d39665d7615b: Status 404 returned error can't find the container with id 5400546f3769a44c814d0b2d4874571008dd2e1221d6e3273238d39665d7615b
Jul 15 20:09:09.148496 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:09.148467    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b8fdfb54d-khn4s"]
Jul 15 20:09:09.150742 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:09:09.150716    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179466fe_25e0_416f_9c70_94c39dc85818.slice/crio-3bc1174246b83637929c185c93a5d4f057ec217a8176ce2deabacc2bcbb911d0 WatchSource:0}: Error finding container 3bc1174246b83637929c185c93a5d4f057ec217a8176ce2deabacc2bcbb911d0: Status 404 returned error can't find the container with id 3bc1174246b83637929c185c93a5d4f057ec217a8176ce2deabacc2bcbb911d0
Jul 15 20:09:09.813848 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:09.813810    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d98d5447-c24vm" event={"ID":"c88ce048-965d-40f0-b7a2-535dc21878f2","Type":"ContainerStarted","Data":"5400546f3769a44c814d0b2d4874571008dd2e1221d6e3273238d39665d7615b"}
Jul 15 20:09:09.815626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:09.815561    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s" event={"ID":"179466fe-25e0-416f-9c70-94c39dc85818","Type":"ContainerStarted","Data":"3bc1174246b83637929c185c93a5d4f057ec217a8176ce2deabacc2bcbb911d0"}
Jul 15 20:09:12.834254 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:12.834215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d98d5447-c24vm" event={"ID":"c88ce048-965d-40f0-b7a2-535dc21878f2","Type":"ContainerStarted","Data":"085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f"}
Jul 15 20:09:12.834783 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:12.834407    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:12.835763 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:12.835739    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s" event={"ID":"179466fe-25e0-416f-9c70-94c39dc85818","Type":"ContainerStarted","Data":"b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220"}
Jul 15 20:09:12.835890 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:12.835820    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:12.854873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:12.854818    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86d98d5447-c24vm" podStartSLOduration=2.091304808 podStartE2EDuration="4.854802429s" totalImagesPullingTime="2.763497621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:09:12.853899482 +0000 UTC m=+634.302272574" watchObservedRunningTime="2026-07-15 20:09:12.854802429 +0000 UTC m=+634.303175541"
Jul 15 20:09:12.873984 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:12.873906    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s" podStartSLOduration=2.142514493 podStartE2EDuration="4.873885875s" totalImagesPullingTime="2.731371382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:09:12.872427001 +0000 UTC m=+634.320800097" watchObservedRunningTime="2026-07-15 20:09:12.873885875 +0000 UTC m=+634.322258971"
Jul 15 20:09:23.842397 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:23.842362    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:23.844780 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:23.842449    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:09:23.927669 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:23.927635    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b8fdfb54d-khn4s"]
Jul 15 20:09:23.927874 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:23.927830    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s" podUID="179466fe-25e0-416f-9c70-94c39dc85818" containerName="manager" containerID="cri-o://b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220" gracePeriod=30
Jul 15 20:09:24.169413 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.169389    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:24.208329 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.208294    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td8b7\" (UniqueName: \"kubernetes.io/projected/179466fe-25e0-416f-9c70-94c39dc85818-kube-api-access-td8b7\") pod \"179466fe-25e0-416f-9c70-94c39dc85818\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") "
Jul 15 20:09:24.208509 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.208345    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/179466fe-25e0-416f-9c70-94c39dc85818-webhook-cert\") pod \"179466fe-25e0-416f-9c70-94c39dc85818\" (UID: \"179466fe-25e0-416f-9c70-94c39dc85818\") "
Jul 15 20:09:24.210385 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.210348    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/179466fe-25e0-416f-9c70-94c39dc85818-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "179466fe-25e0-416f-9c70-94c39dc85818" (UID: "179466fe-25e0-416f-9c70-94c39dc85818"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:09:24.210519 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.210495    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/179466fe-25e0-416f-9c70-94c39dc85818-kube-api-access-td8b7" (OuterVolumeSpecName: "kube-api-access-td8b7") pod "179466fe-25e0-416f-9c70-94c39dc85818" (UID: "179466fe-25e0-416f-9c70-94c39dc85818"). InnerVolumeSpecName "kube-api-access-td8b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:09:24.309001 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.308930    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td8b7\" (UniqueName: \"kubernetes.io/projected/179466fe-25e0-416f-9c70-94c39dc85818-kube-api-access-td8b7\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:09:24.309001 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.308962    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/179466fe-25e0-416f-9c70-94c39dc85818-webhook-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:09:24.892120 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.892087    2562 generic.go:356] "Generic (PLEG): container finished" podID="179466fe-25e0-416f-9c70-94c39dc85818" containerID="b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220" exitCode=0
Jul 15 20:09:24.892570 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.892127    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s"
Jul 15 20:09:24.892570 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.892177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s" event={"ID":"179466fe-25e0-416f-9c70-94c39dc85818","Type":"ContainerDied","Data":"b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220"}
Jul 15 20:09:24.892570 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.892228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8fdfb54d-khn4s" event={"ID":"179466fe-25e0-416f-9c70-94c39dc85818","Type":"ContainerDied","Data":"3bc1174246b83637929c185c93a5d4f057ec217a8176ce2deabacc2bcbb911d0"}
Jul 15 20:09:24.892570 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.892251    2562 scope.go:117] "RemoveContainer" containerID="b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220"
Jul 15 20:09:24.901316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.901300    2562 scope.go:117] "RemoveContainer" containerID="b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220"
Jul 15 20:09:24.901572 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:09:24.901549    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220\": container with ID starting with b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220 not found: ID does not exist" containerID="b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220"
Jul 15 20:09:24.901665 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.901579    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220"} err="failed to get container status \"b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220\": rpc error: code = NotFound desc = could not find container \"b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220\": container with ID starting with b7a152f04ce8b29add4be034ffa5e2cccd9148d41d1d6e6fbc2d768ca5987220 not found: ID does not exist"
Jul 15 20:09:24.917813 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.917782    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b8fdfb54d-khn4s"]
Jul 15 20:09:24.927365 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:24.927337    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b8fdfb54d-khn4s"]
Jul 15 20:09:25.077860 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:25.077830    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="179466fe-25e0-416f-9c70-94c39dc85818" path="/var/lib/kubelet/pods/179466fe-25e0-416f-9c70-94c39dc85818/volumes"
Jul 15 20:09:44.726514 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.726478    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"]
Jul 15 20:09:44.727003 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.726836    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="179466fe-25e0-416f-9c70-94c39dc85818" containerName="manager"
Jul 15 20:09:44.727003 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.726847    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="179466fe-25e0-416f-9c70-94c39dc85818" containerName="manager"
Jul 15 20:09:44.727003 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.726911    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="179466fe-25e0-416f-9c70-94c39dc85818" containerName="manager"
Jul 15 20:09:44.731134 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.731112    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.733789 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.733765    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:09:44.733789 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.733799    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z7fnv\""
Jul 15 20:09:44.734012 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.733766    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 20:09:44.734803 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.734786    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 20:09:44.740777 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.740753    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"]
Jul 15 20:09:44.792345 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.792309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p8m2\" (UniqueName: \"kubernetes.io/projected/855acac0-fd56-45f7-8660-9e330c20f48e-kube-api-access-8p8m2\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.792536 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.792358    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/855acac0-fd56-45f7-8660-9e330c20f48e-maas-api-tls\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.792536 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.792446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/855acac0-fd56-45f7-8660-9e330c20f48e-openshift-service-ca\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.893747 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.893713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/855acac0-fd56-45f7-8660-9e330c20f48e-openshift-service-ca\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.893953 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.893772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p8m2\" (UniqueName: \"kubernetes.io/projected/855acac0-fd56-45f7-8660-9e330c20f48e-kube-api-access-8p8m2\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.893953 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.893802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/855acac0-fd56-45f7-8660-9e330c20f48e-maas-api-tls\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.894516 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.894496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/855acac0-fd56-45f7-8660-9e330c20f48e-openshift-service-ca\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.896301 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.896284    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/855acac0-fd56-45f7-8660-9e330c20f48e-maas-api-tls\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:44.905938 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:44.905916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p8m2\" (UniqueName: \"kubernetes.io/projected/855acac0-fd56-45f7-8660-9e330c20f48e-kube-api-access-8p8m2\") pod \"maas-api-54f47c8df8-h92lb\" (UID: \"855acac0-fd56-45f7-8660-9e330c20f48e\") " pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:45.042378 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.042294    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:45.204474 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.204436    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"]
Jul 15 20:09:45.211059 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:09:45.211022    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855acac0_fd56_45f7_8660_9e330c20f48e.slice/crio-10c995924de8e99b596280caa64f89b541e672b63b9f20291984f8fe937dccab WatchSource:0}: Error finding container 10c995924de8e99b596280caa64f89b541e672b63b9f20291984f8fe937dccab: Status 404 returned error can't find the container with id 10c995924de8e99b596280caa64f89b541e672b63b9f20291984f8fe937dccab
Jul 15 20:09:45.651706 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.651667    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"]
Jul 15 20:09:45.655226 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.655203    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.661130 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.661106    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 20:09:45.661259 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.661170    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ccvv5\""
Jul 15 20:09:45.674435 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.674406    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"]
Jul 15 20:09:45.802000 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.801951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5606231-994f-4b2b-9c01-4d7bd6b5d0a2-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-6vg7p\" (UID: \"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.802349 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.802106    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t6nf\" (UniqueName: \"kubernetes.io/projected/c5606231-994f-4b2b-9c01-4d7bd6b5d0a2-kube-api-access-4t6nf\") pod \"payload-pre-processing-74584bb9f8-6vg7p\" (UID: \"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.903470 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.903371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5606231-994f-4b2b-9c01-4d7bd6b5d0a2-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-6vg7p\" (UID: \"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.903642 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.903476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t6nf\" (UniqueName: \"kubernetes.io/projected/c5606231-994f-4b2b-9c01-4d7bd6b5d0a2-kube-api-access-4t6nf\") pod \"payload-pre-processing-74584bb9f8-6vg7p\" (UID: \"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.904189 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.904162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5606231-994f-4b2b-9c01-4d7bd6b5d0a2-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-6vg7p\" (UID: \"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.915342 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.915312    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t6nf\" (UniqueName: \"kubernetes.io/projected/c5606231-994f-4b2b-9c01-4d7bd6b5d0a2-kube-api-access-4t6nf\") pod \"payload-pre-processing-74584bb9f8-6vg7p\" (UID: \"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.966363 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.966320    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:45.991411 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:45.991379    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb" event={"ID":"855acac0-fd56-45f7-8660-9e330c20f48e","Type":"ContainerStarted","Data":"10c995924de8e99b596280caa64f89b541e672b63b9f20291984f8fe937dccab"}
Jul 15 20:09:46.129184 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:46.129149    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"]
Jul 15 20:09:46.132902 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:09:46.132875    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5606231_994f_4b2b_9c01_4d7bd6b5d0a2.slice/crio-529b8df3311998fe9e5499119c5e4256aa605ab5685f13a84bf8dd1c08b5dab1 WatchSource:0}: Error finding container 529b8df3311998fe9e5499119c5e4256aa605ab5685f13a84bf8dd1c08b5dab1: Status 404 returned error can't find the container with id 529b8df3311998fe9e5499119c5e4256aa605ab5685f13a84bf8dd1c08b5dab1
Jul 15 20:09:46.998890 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:46.998825    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p" event={"ID":"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2","Type":"ContainerStarted","Data":"529b8df3311998fe9e5499119c5e4256aa605ab5685f13a84bf8dd1c08b5dab1"}
Jul 15 20:09:48.004907 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:48.004869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p" event={"ID":"c5606231-994f-4b2b-9c01-4d7bd6b5d0a2","Type":"ContainerStarted","Data":"54fd1a1a1d4167d9e46c3f14a658a0b83a47934f6c020c6fa9222226a2d7cd7d"}
Jul 15 20:09:48.005382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:48.004934    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:09:48.006419 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:48.006391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb" event={"ID":"855acac0-fd56-45f7-8660-9e330c20f48e","Type":"ContainerStarted","Data":"3bc69f8f4def84ba35e6ce816782ed2cb4a5bb0809a10b89e49d4ae44b735088"}
Jul 15 20:09:48.006532 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:48.006521    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:48.026021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:48.025945    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p" podStartSLOduration=1.5318482919999998 podStartE2EDuration="3.025925216s" totalImagesPullingTime="1.494076924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:09:48.025032315 +0000 UTC m=+669.473405411" watchObservedRunningTime="2026-07-15 20:09:48.025925216 +0000 UTC m=+669.474298365"
Jul 15 20:09:48.046753 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:48.046688    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb" podStartSLOduration=2.145283951 podStartE2EDuration="4.046670341s" totalImagesPullingTime="1.90138639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:09:48.04594659 +0000 UTC m=+669.494319689" watchObservedRunningTime="2026-07-15 20:09:48.046670341 +0000 UTC m=+669.495043438"
Jul 15 20:09:54.016022 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:54.015988    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54f47c8df8-h92lb"
Jul 15 20:09:59.012861 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:09:59.012818    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6vg7p"
Jul 15 20:10:27.966522 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:27.966489    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"]
Jul 15 20:10:27.971095 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:27.971073    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:27.982487 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:27.982456    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 15 20:10:27.982681 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:27.982490    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 20:10:27.982681 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:27.982553    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 20:10:27.982681 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:27.982461    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rkl7w\""
Jul 15 20:10:28.001214 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.001187    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"]
Jul 15 20:10:28.059425 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.059387    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd7c479c-6664-464e-a5d0-b9f3ca29459d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.059625 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.059440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.059625 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.059462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.059625 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.059483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.059625 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.059499    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.059625 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.059568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7k2s\" (UniqueName: \"kubernetes.io/projected/fd7c479c-6664-464e-a5d0-b9f3ca29459d-kube-api-access-k7k2s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.160745 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.160706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd7c479c-6664-464e-a5d0-b9f3ca29459d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.160929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.160755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.160929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.160774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.160929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.160793    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.160929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.160817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.160929 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.160849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7k2s\" (UniqueName: \"kubernetes.io/projected/fd7c479c-6664-464e-a5d0-b9f3ca29459d-kube-api-access-k7k2s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.161229 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.161208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.161288 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.161248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.161345 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.161324    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.163152 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.163129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd7c479c-6664-464e-a5d0-b9f3ca29459d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.163311 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.163295    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd7c479c-6664-464e-a5d0-b9f3ca29459d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.169688 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.169664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7k2s\" (UniqueName: \"kubernetes.io/projected/fd7c479c-6664-464e-a5d0-b9f3ca29459d-kube-api-access-k7k2s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4s5zn\" (UID: \"fd7c479c-6664-464e-a5d0-b9f3ca29459d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.281040 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.280924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:28.423386 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.423358    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"]
Jul 15 20:10:28.425533 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:10:28.425504    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7c479c_6664_464e_a5d0_b9f3ca29459d.slice/crio-841015ce60b97d35bd538faa9481b151da7e9e1a9d60811ce162d4f2ce7b561e WatchSource:0}: Error finding container 841015ce60b97d35bd538faa9481b151da7e9e1a9d60811ce162d4f2ce7b561e: Status 404 returned error can't find the container with id 841015ce60b97d35bd538faa9481b151da7e9e1a9d60811ce162d4f2ce7b561e
Jul 15 20:10:28.426815 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:28.426799    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:10:29.190958 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:29.190904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn" event={"ID":"fd7c479c-6664-464e-a5d0-b9f3ca29459d","Type":"ContainerStarted","Data":"841015ce60b97d35bd538faa9481b151da7e9e1a9d60811ce162d4f2ce7b561e"}
Jul 15 20:10:30.197690 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:30.197649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn" event={"ID":"fd7c479c-6664-464e-a5d0-b9f3ca29459d","Type":"ContainerStarted","Data":"759b0c9f3df07001ad61b923ae6e08c8e960482ca5d5a799bfad4c8591b939fb"}
Jul 15 20:10:30.216782 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:30.216731    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn" podStartSLOduration=2.199983537 podStartE2EDuration="3.216716462s" totalImagesPullingTime="1.016732925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:10:30.216402062 +0000 UTC m=+711.664775159" watchObservedRunningTime="2026-07-15 20:10:30.216716462 +0000 UTC m=+711.665089558"
Jul 15 20:10:38.281743 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:38.281708    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:38.282255 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:38.281777    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:38.294365 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:38.294335    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:39.247199 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:39.247171    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4s5zn"
Jul 15 20:10:47.969025 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:47.968986    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"]
Jul 15 20:10:47.972221 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:47.972201    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:47.975509 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:47.975487    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 15 20:10:47.986160 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:47.986130    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"]
Jul 15 20:10:48.150931 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.150899    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.150931 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.150934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg8dk\" (UniqueName: \"kubernetes.io/projected/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-kube-api-access-rg8dk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.151210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.151013    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.151210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.151033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.151210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.151060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.151210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.151098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.251791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.251674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.251791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.251741    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.251791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.251791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.252131 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.251938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.252131 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.252017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg8dk\" (UniqueName: \"kubernetes.io/projected/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-kube-api-access-rg8dk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.252131 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.252075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.252287 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.252209    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.252287 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.252224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.252387 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.252368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.254114 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.254095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.254360 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.254341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.263104 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.263077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg8dk\" (UniqueName: \"kubernetes.io/projected/ca8a4d34-1a12-48e7-b39b-443bd1b53bba-kube-api-access-rg8dk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh\" (UID: \"ca8a4d34-1a12-48e7-b39b-443bd1b53bba\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.282577 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.282543    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:48.427399 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:48.427361    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"]
Jul 15 20:10:48.430369 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:10:48.430339    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca8a4d34_1a12_48e7_b39b_443bd1b53bba.slice/crio-4aff4e3c34035ab561b025ea8d4e6d0ffa32ed4d48f170b386e9a392af59613b WatchSource:0}: Error finding container 4aff4e3c34035ab561b025ea8d4e6d0ffa32ed4d48f170b386e9a392af59613b: Status 404 returned error can't find the container with id 4aff4e3c34035ab561b025ea8d4e6d0ffa32ed4d48f170b386e9a392af59613b
Jul 15 20:10:49.282407 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:49.282371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh" event={"ID":"ca8a4d34-1a12-48e7-b39b-443bd1b53bba","Type":"ContainerStarted","Data":"5117ef99ac174df8f0158c1ff1a2f2c64f2426eb4492c62ae738401c854df5b4"}
Jul 15 20:10:49.282911 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:49.282434    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh" event={"ID":"ca8a4d34-1a12-48e7-b39b-443bd1b53bba","Type":"ContainerStarted","Data":"4aff4e3c34035ab561b025ea8d4e6d0ffa32ed4d48f170b386e9a392af59613b"}
Jul 15 20:10:49.302415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:49.302353    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh" podStartSLOduration=2.074200042 podStartE2EDuration="2.302336461s" totalImagesPullingTime="228.136419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:10:49.300652929 +0000 UTC m=+730.749026026" watchObservedRunningTime="2026-07-15 20:10:49.302336461 +0000 UTC m=+730.750709556"
Jul 15 20:10:53.244415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.244374    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"]
Jul 15 20:10:53.248101 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.248075    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.251531 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.251500    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 15 20:10:53.264696 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.264671    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"]
Jul 15 20:10:53.405496 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.405454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q47jk\" (UniqueName: \"kubernetes.io/projected/100badf6-fba8-4032-b26f-64b1b6ad9d0f-kube-api-access-q47jk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.405671 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.405590    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.405671 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.405641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.405796 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.405676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.405796 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.405719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.405796 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.405765    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/100badf6-fba8-4032-b26f-64b1b6ad9d0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507133 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507036    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/100badf6-fba8-4032-b26f-64b1b6ad9d0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507133 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q47jk\" (UniqueName: \"kubernetes.io/projected/100badf6-fba8-4032-b26f-64b1b6ad9d0f-kube-api-access-q47jk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507280    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507721 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507838 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.507838 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.507731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.509495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.509463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/100badf6-fba8-4032-b26f-64b1b6ad9d0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.509652 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.509634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/100badf6-fba8-4032-b26f-64b1b6ad9d0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.524726 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.524701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q47jk\" (UniqueName: \"kubernetes.io/projected/100badf6-fba8-4032-b26f-64b1b6ad9d0f-kube-api-access-q47jk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n45v7\" (UID: \"100badf6-fba8-4032-b26f-64b1b6ad9d0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.558636 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.558593    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:10:53.701238 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:53.701208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"]
Jul 15 20:10:53.702472 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:10:53.702450    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100badf6_fba8_4032_b26f_64b1b6ad9d0f.slice/crio-7c1184b6c9b3c995f2c2987ddc9df5058a959a470b6044caf81a170865e7a814 WatchSource:0}: Error finding container 7c1184b6c9b3c995f2c2987ddc9df5058a959a470b6044caf81a170865e7a814: Status 404 returned error can't find the container with id 7c1184b6c9b3c995f2c2987ddc9df5058a959a470b6044caf81a170865e7a814
Jul 15 20:10:54.304859 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:54.304771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7" event={"ID":"100badf6-fba8-4032-b26f-64b1b6ad9d0f","Type":"ContainerStarted","Data":"4c5d736253eb1834c99681c01bbf1958a63ad127e755f0d2a44e1b61db4fd08b"}
Jul 15 20:10:54.304859 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:54.304835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7" event={"ID":"100badf6-fba8-4032-b26f-64b1b6ad9d0f","Type":"ContainerStarted","Data":"7c1184b6c9b3c995f2c2987ddc9df5058a959a470b6044caf81a170865e7a814"}
Jul 15 20:10:54.326667 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:54.326579    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7" podStartSLOduration=1.076726504 podStartE2EDuration="1.326562457s" totalImagesPullingTime="249.835953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:10:54.325822809 +0000 UTC m=+735.774195906" watchObservedRunningTime="2026-07-15 20:10:54.326562457 +0000 UTC m=+735.774935554"
Jul 15 20:10:58.283143 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:58.283111    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:58.283611 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:58.283208    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:58.295946 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:58.295914    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:10:58.334985 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:10:58.334943    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh"
Jul 15 20:11:03.558888 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:11:03.558843    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:11:03.559382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:11:03.558933    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:11:03.571594 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:11:03.571563    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:11:04.363631 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:11:04.363597    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n45v7"
Jul 15 20:13:30.611727 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:30.611626    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d98d5447-c24vm"]
Jul 15 20:13:30.612367 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:30.611854    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86d98d5447-c24vm" podUID="c88ce048-965d-40f0-b7a2-535dc21878f2" containerName="manager" containerID="cri-o://085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f" gracePeriod=30
Jul 15 20:13:30.863056 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:30.862963    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:13:30.945382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:30.945353    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c88ce048-965d-40f0-b7a2-535dc21878f2-webhook-cert\") pod \"c88ce048-965d-40f0-b7a2-535dc21878f2\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") "
Jul 15 20:13:30.945546 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:30.945425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcvgz\" (UniqueName: \"kubernetes.io/projected/c88ce048-965d-40f0-b7a2-535dc21878f2-kube-api-access-dcvgz\") pod \"c88ce048-965d-40f0-b7a2-535dc21878f2\" (UID: \"c88ce048-965d-40f0-b7a2-535dc21878f2\") "
Jul 15 20:13:30.947285 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:30.947252    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c88ce048-965d-40f0-b7a2-535dc21878f2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c88ce048-965d-40f0-b7a2-535dc21878f2" (UID: "c88ce048-965d-40f0-b7a2-535dc21878f2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:13:30.947394 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:30.947321    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c88ce048-965d-40f0-b7a2-535dc21878f2-kube-api-access-dcvgz" (OuterVolumeSpecName: "kube-api-access-dcvgz") pod "c88ce048-965d-40f0-b7a2-535dc21878f2" (UID: "c88ce048-965d-40f0-b7a2-535dc21878f2"). InnerVolumeSpecName "kube-api-access-dcvgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:13:31.035526 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.035488    2562 generic.go:356] "Generic (PLEG): container finished" podID="c88ce048-965d-40f0-b7a2-535dc21878f2" containerID="085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f" exitCode=0
Jul 15 20:13:31.035526 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.035532    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d98d5447-c24vm"
Jul 15 20:13:31.035722 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.035571    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d98d5447-c24vm" event={"ID":"c88ce048-965d-40f0-b7a2-535dc21878f2","Type":"ContainerDied","Data":"085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f"}
Jul 15 20:13:31.035722 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.035631    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d98d5447-c24vm" event={"ID":"c88ce048-965d-40f0-b7a2-535dc21878f2","Type":"ContainerDied","Data":"5400546f3769a44c814d0b2d4874571008dd2e1221d6e3273238d39665d7615b"}
Jul 15 20:13:31.035722 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.035654    2562 scope.go:117] "RemoveContainer" containerID="085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f"
Jul 15 20:13:31.044585 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.044567    2562 scope.go:117] "RemoveContainer" containerID="085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f"
Jul 15 20:13:31.044848 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:13:31.044828    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f\": container with ID starting with 085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f not found: ID does not exist" containerID="085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f"
Jul 15 20:13:31.044895 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.044857    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f"} err="failed to get container status \"085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f\": rpc error: code = NotFound desc = could not find container \"085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f\": container with ID starting with 085b16020cc7fd51b1af9b984898ea17e80dbfb615f1c01175b979af75e28b1f not found: ID does not exist"
Jul 15 20:13:31.046279 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.046264    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c88ce048-965d-40f0-b7a2-535dc21878f2-webhook-cert\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:13:31.046330 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.046284    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcvgz\" (UniqueName: \"kubernetes.io/projected/c88ce048-965d-40f0-b7a2-535dc21878f2-kube-api-access-dcvgz\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:13:31.060437 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.060385    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d98d5447-c24vm"]
Jul 15 20:13:31.065500 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.065479    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86d98d5447-c24vm"]
Jul 15 20:13:31.083294 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.079780    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c88ce048-965d-40f0-b7a2-535dc21878f2" path="/var/lib/kubelet/pods/c88ce048-965d-40f0-b7a2-535dc21878f2/volumes"
Jul 15 20:13:31.880208 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.880169    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86d98d5447-db7qh"]
Jul 15 20:13:31.880565 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.880543    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c88ce048-965d-40f0-b7a2-535dc21878f2" containerName="manager"
Jul 15 20:13:31.880565 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.880553    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c88ce048-965d-40f0-b7a2-535dc21878f2" containerName="manager"
Jul 15 20:13:31.880632 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.880610    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c88ce048-965d-40f0-b7a2-535dc21878f2" containerName="manager"
Jul 15 20:13:31.885053 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.884990    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:31.888073 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.888049    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 20:13:31.888187 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.888074    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 20:13:31.888943 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.888927    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x9wjp\""
Jul 15 20:13:31.895740 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.895714    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d98d5447-db7qh"]
Jul 15 20:13:31.955888 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.955855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924198f2-21e9-49fa-8310-87cdfa824e2d-webhook-cert\") pod \"maas-controller-86d98d5447-db7qh\" (UID: \"924198f2-21e9-49fa-8310-87cdfa824e2d\") " pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:31.956119 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:31.955919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p77m\" (UniqueName: \"kubernetes.io/projected/924198f2-21e9-49fa-8310-87cdfa824e2d-kube-api-access-2p77m\") pod \"maas-controller-86d98d5447-db7qh\" (UID: \"924198f2-21e9-49fa-8310-87cdfa824e2d\") " pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:32.056984 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:32.056945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924198f2-21e9-49fa-8310-87cdfa824e2d-webhook-cert\") pod \"maas-controller-86d98d5447-db7qh\" (UID: \"924198f2-21e9-49fa-8310-87cdfa824e2d\") " pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:32.057190 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:32.057033    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p77m\" (UniqueName: \"kubernetes.io/projected/924198f2-21e9-49fa-8310-87cdfa824e2d-kube-api-access-2p77m\") pod \"maas-controller-86d98d5447-db7qh\" (UID: \"924198f2-21e9-49fa-8310-87cdfa824e2d\") " pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:32.059556 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:32.059523    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924198f2-21e9-49fa-8310-87cdfa824e2d-webhook-cert\") pod \"maas-controller-86d98d5447-db7qh\" (UID: \"924198f2-21e9-49fa-8310-87cdfa824e2d\") " pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:32.067094 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:32.067070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p77m\" (UniqueName: \"kubernetes.io/projected/924198f2-21e9-49fa-8310-87cdfa824e2d-kube-api-access-2p77m\") pod \"maas-controller-86d98d5447-db7qh\" (UID: \"924198f2-21e9-49fa-8310-87cdfa824e2d\") " pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:32.195831 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:32.195792    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:32.324911 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:32.324888    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d98d5447-db7qh"]
Jul 15 20:13:32.327053 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:13:32.327027    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924198f2_21e9_49fa_8310_87cdfa824e2d.slice/crio-5d56020d44ca5328cb942f499996a1f8f77318d7dac778c1c8ba9550940c4d1d WatchSource:0}: Error finding container 5d56020d44ca5328cb942f499996a1f8f77318d7dac778c1c8ba9550940c4d1d: Status 404 returned error can't find the container with id 5d56020d44ca5328cb942f499996a1f8f77318d7dac778c1c8ba9550940c4d1d
Jul 15 20:13:33.047876 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:33.047842    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d98d5447-db7qh" event={"ID":"924198f2-21e9-49fa-8310-87cdfa824e2d","Type":"ContainerStarted","Data":"35c6ed946b6986501de7510ff15da99e165b7f2c01a173c636ee7414ce2fa322"}
Jul 15 20:13:33.048343 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:33.047900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d98d5447-db7qh" event={"ID":"924198f2-21e9-49fa-8310-87cdfa824e2d","Type":"ContainerStarted","Data":"5d56020d44ca5328cb942f499996a1f8f77318d7dac778c1c8ba9550940c4d1d"}
Jul 15 20:13:33.048343 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:33.047992    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:13:33.067634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:33.067584    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86d98d5447-db7qh" podStartSLOduration=1.506618988 podStartE2EDuration="2.067565632s" totalImagesPullingTime="560.946644ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:13:33.066938385 +0000 UTC m=+894.515311491" watchObservedRunningTime="2026-07-15 20:13:33.067565632 +0000 UTC m=+894.515938731"
Jul 15 20:13:44.055099 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:13:44.055068    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86d98d5447-db7qh"
Jul 15 20:25:59.944188 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:25:59.944154    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"]
Jul 15 20:25:59.947432 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:25:59.947409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:25:59.952556 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:25:59.952536    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-609ae16c-serving-cert\""
Jul 15 20:25:59.986330 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:25:59.986302    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"]
Jul 15 20:26:00.075979 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.075942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11931165-d152-4ff9-a031-2fdaa1f7ffb4-openshift-service-ca\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.076109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.076003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.076109 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.076049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhfkl\" (UniqueName: \"kubernetes.io/projected/11931165-d152-4ff9-a031-2fdaa1f7ffb4-kube-api-access-hhfkl\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.177327 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.177289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhfkl\" (UniqueName: \"kubernetes.io/projected/11931165-d152-4ff9-a031-2fdaa1f7ffb4-kube-api-access-hhfkl\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.177508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.177389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11931165-d152-4ff9-a031-2fdaa1f7ffb4-openshift-service-ca\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.177508 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.177434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.177633 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:26:00.177564    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-serving-cert: secret "maas-api-e2e-ait-609ae16c-serving-cert" not found
Jul 15 20:26:00.177695 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:26:00.177674    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls podName:11931165-d152-4ff9-a031-2fdaa1f7ffb4 nodeName:}" failed. No retries permitted until 2026-07-15 20:26:00.677650671 +0000 UTC m=+1642.126023746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls") pod "maas-api-e2e-ait-609ae16c-776bd99c96-r7r47" (UID: "11931165-d152-4ff9-a031-2fdaa1f7ffb4") : secret "maas-api-e2e-ait-609ae16c-serving-cert" not found
Jul 15 20:26:00.178082 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.178062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11931165-d152-4ff9-a031-2fdaa1f7ffb4-openshift-service-ca\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.191481 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.191451    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhfkl\" (UniqueName: \"kubernetes.io/projected/11931165-d152-4ff9-a031-2fdaa1f7ffb4-kube-api-access-hhfkl\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.681802 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.681767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.684276 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.684243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls\") pod \"maas-api-e2e-ait-609ae16c-776bd99c96-r7r47\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:00.861878 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:00.861848    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:01.036800 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:01.036768    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"]
Jul 15 20:26:01.040070 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:26:01.040042    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11931165_d152_4ff9_a031_2fdaa1f7ffb4.slice/crio-494894b7d2fd38aff8bbf9023a7704dce56ec68022596620abe7d33aab2eb1ae WatchSource:0}: Error finding container 494894b7d2fd38aff8bbf9023a7704dce56ec68022596620abe7d33aab2eb1ae: Status 404 returned error can't find the container with id 494894b7d2fd38aff8bbf9023a7704dce56ec68022596620abe7d33aab2eb1ae
Jul 15 20:26:01.041429 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:01.041408    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:26:01.505335 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:01.505304    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47" event={"ID":"11931165-d152-4ff9-a031-2fdaa1f7ffb4","Type":"ContainerStarted","Data":"494894b7d2fd38aff8bbf9023a7704dce56ec68022596620abe7d33aab2eb1ae"}
Jul 15 20:26:02.511332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:02.511298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47" event={"ID":"11931165-d152-4ff9-a031-2fdaa1f7ffb4","Type":"ContainerStarted","Data":"de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3"}
Jul 15 20:26:02.511731 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:02.511425    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:26:02.547562 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:02.547515    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47" podStartSLOduration=2.267659866 podStartE2EDuration="3.547498187s" totalImagesPullingTime="1.279838321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:26:02.545227826 +0000 UTC m=+1643.993600922" watchObservedRunningTime="2026-07-15 20:26:02.547498187 +0000 UTC m=+1643.995871325"
Jul 15 20:26:08.521280 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:26:08.521248    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:27:15.175761 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.175726    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"]
Jul 15 20:27:15.176316 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.175987    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47" podUID="11931165-d152-4ff9-a031-2fdaa1f7ffb4" containerName="maas-api" containerID="cri-o://de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3" gracePeriod=30
Jul 15 20:27:15.423235 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.423206    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:27:15.456572 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.456541    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11931165-d152-4ff9-a031-2fdaa1f7ffb4-openshift-service-ca\") pod \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") "
Jul 15 20:27:15.456756 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.456610    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls\") pod \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") "
Jul 15 20:27:15.456756 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.456671    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhfkl\" (UniqueName: \"kubernetes.io/projected/11931165-d152-4ff9-a031-2fdaa1f7ffb4-kube-api-access-hhfkl\") pod \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\" (UID: \"11931165-d152-4ff9-a031-2fdaa1f7ffb4\") "
Jul 15 20:27:15.456949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.456925    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11931165-d152-4ff9-a031-2fdaa1f7ffb4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11931165-d152-4ff9-a031-2fdaa1f7ffb4" (UID: "11931165-d152-4ff9-a031-2fdaa1f7ffb4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:27:15.458908 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.458873    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11931165-d152-4ff9-a031-2fdaa1f7ffb4" (UID: "11931165-d152-4ff9-a031-2fdaa1f7ffb4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:27:15.458908 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.458884    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11931165-d152-4ff9-a031-2fdaa1f7ffb4-kube-api-access-hhfkl" (OuterVolumeSpecName: "kube-api-access-hhfkl") pod "11931165-d152-4ff9-a031-2fdaa1f7ffb4" (UID: "11931165-d152-4ff9-a031-2fdaa1f7ffb4"). InnerVolumeSpecName "kube-api-access-hhfkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:27:15.557470 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.557433    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhfkl\" (UniqueName: \"kubernetes.io/projected/11931165-d152-4ff9-a031-2fdaa1f7ffb4-kube-api-access-hhfkl\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:27:15.557470 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.557465    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11931165-d152-4ff9-a031-2fdaa1f7ffb4-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:27:15.557470 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.557475    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11931165-d152-4ff9-a031-2fdaa1f7ffb4-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:27:15.854024 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.853922    2562 generic.go:356] "Generic (PLEG): container finished" podID="11931165-d152-4ff9-a031-2fdaa1f7ffb4" containerID="de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3" exitCode=0
Jul 15 20:27:15.854024 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.853975    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47" event={"ID":"11931165-d152-4ff9-a031-2fdaa1f7ffb4","Type":"ContainerDied","Data":"de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3"}
Jul 15 20:27:15.854204 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.854032    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47" event={"ID":"11931165-d152-4ff9-a031-2fdaa1f7ffb4","Type":"ContainerDied","Data":"494894b7d2fd38aff8bbf9023a7704dce56ec68022596620abe7d33aab2eb1ae"}
Jul 15 20:27:15.854204 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.854054    2562 scope.go:117] "RemoveContainer" containerID="de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3"
Jul 15 20:27:15.854204 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.853985    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"
Jul 15 20:27:15.864677 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.864656    2562 scope.go:117] "RemoveContainer" containerID="de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3"
Jul 15 20:27:15.865433 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:27:15.865393    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3\": container with ID starting with de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3 not found: ID does not exist" containerID="de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3"
Jul 15 20:27:15.865756 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.865439    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3"} err="failed to get container status \"de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3\": rpc error: code = NotFound desc = could not find container \"de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3\": container with ID starting with de6bd18ffc197585572076e00dc2789fe806b72acdeb756828c33ef5f85c4ef3 not found: ID does not exist"
Jul 15 20:27:15.876800 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.876769    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"]
Jul 15 20:27:15.879500 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:15.879119    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-609ae16c-776bd99c96-r7r47"]
Jul 15 20:27:17.078476 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:17.078441    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11931165-d152-4ff9-a031-2fdaa1f7ffb4" path="/var/lib/kubelet/pods/11931165-d152-4ff9-a031-2fdaa1f7ffb4/volumes"
Jul 15 20:27:32.249335 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.249301    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"]
Jul 15 20:27:32.249694 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.249660    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11931165-d152-4ff9-a031-2fdaa1f7ffb4" containerName="maas-api"
Jul 15 20:27:32.249694 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.249670    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="11931165-d152-4ff9-a031-2fdaa1f7ffb4" containerName="maas-api"
Jul 15 20:27:32.249767 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.249756    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="11931165-d152-4ff9-a031-2fdaa1f7ffb4" containerName="maas-api"
Jul 15 20:27:32.253919 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.253903    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.278900 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.278867    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"]
Jul 15 20:27:32.282861 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.282841    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-be7332c6-serving-cert\""
Jul 15 20:27:32.410939 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.410903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2016e952-133e-4314-a18a-40a9033a0317-openshift-service-ca\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.411146 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.410986    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.411146 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.411100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7rc9\" (UniqueName: \"kubernetes.io/projected/2016e952-133e-4314-a18a-40a9033a0317-kube-api-access-s7rc9\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.511742 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.511646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2016e952-133e-4314-a18a-40a9033a0317-openshift-service-ca\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.511742 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.511717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.512067 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.511768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7rc9\" (UniqueName: \"kubernetes.io/projected/2016e952-133e-4314-a18a-40a9033a0317-kube-api-access-s7rc9\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.512067 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:27:32.511834    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-serving-cert: secret "maas-api-e2e-migrate-be7332c6-serving-cert" not found
Jul 15 20:27:32.512067 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:27:32.511907    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls podName:2016e952-133e-4314-a18a-40a9033a0317 nodeName:}" failed. No retries permitted until 2026-07-15 20:27:33.01189205 +0000 UTC m=+1734.460265126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls") pod "maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5" (UID: "2016e952-133e-4314-a18a-40a9033a0317") : secret "maas-api-e2e-migrate-be7332c6-serving-cert" not found
Jul 15 20:27:32.512511 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.512488    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2016e952-133e-4314-a18a-40a9033a0317-openshift-service-ca\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:32.555373 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:32.555337    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7rc9\" (UniqueName: \"kubernetes.io/projected/2016e952-133e-4314-a18a-40a9033a0317-kube-api-access-s7rc9\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:33.016677 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.016643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:33.019034 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.019016    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls\") pod \"maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:33.163119 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.163085    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:33.526294 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.526269    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"]
Jul 15 20:27:33.528994 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:27:33.528937    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2016e952_133e_4314_a18a_40a9033a0317.slice/crio-51d0ce8005be1c4b7a5ed046a1473cde9f8637894b2f965ccc37e9a8cf89d735 WatchSource:0}: Error finding container 51d0ce8005be1c4b7a5ed046a1473cde9f8637894b2f965ccc37e9a8cf89d735: Status 404 returned error can't find the container with id 51d0ce8005be1c4b7a5ed046a1473cde9f8637894b2f965ccc37e9a8cf89d735
Jul 15 20:27:33.937899 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.937859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5" event={"ID":"2016e952-133e-4314-a18a-40a9033a0317","Type":"ContainerStarted","Data":"bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385"}
Jul 15 20:27:33.938050 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.937913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5" event={"ID":"2016e952-133e-4314-a18a-40a9033a0317","Type":"ContainerStarted","Data":"51d0ce8005be1c4b7a5ed046a1473cde9f8637894b2f965ccc37e9a8cf89d735"}
Jul 15 20:27:33.938050 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.937932    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:27:33.978042 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:33.977986    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5" podStartSLOduration=1.629030239 podStartE2EDuration="1.977950769s" totalImagesPullingTime="348.92053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:27:33.977545486 +0000 UTC m=+1735.425918582" watchObservedRunningTime="2026-07-15 20:27:33.977950769 +0000 UTC m=+1735.426323865"
Jul 15 20:27:39.952880 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:27:39.952848    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:28:37.160269 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.160233    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"]
Jul 15 20:28:37.160748 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.160503    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5" podUID="2016e952-133e-4314-a18a-40a9033a0317" containerName="maas-api" containerID="cri-o://bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385" gracePeriod=30
Jul 15 20:28:37.408282 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.408253    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:28:37.511566 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.511530    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2016e952-133e-4314-a18a-40a9033a0317-openshift-service-ca\") pod \"2016e952-133e-4314-a18a-40a9033a0317\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") "
Jul 15 20:28:37.511775 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.511608    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls\") pod \"2016e952-133e-4314-a18a-40a9033a0317\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") "
Jul 15 20:28:37.511775 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.511652    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7rc9\" (UniqueName: \"kubernetes.io/projected/2016e952-133e-4314-a18a-40a9033a0317-kube-api-access-s7rc9\") pod \"2016e952-133e-4314-a18a-40a9033a0317\" (UID: \"2016e952-133e-4314-a18a-40a9033a0317\") "
Jul 15 20:28:37.512027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.511998    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2016e952-133e-4314-a18a-40a9033a0317-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2016e952-133e-4314-a18a-40a9033a0317" (UID: "2016e952-133e-4314-a18a-40a9033a0317"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:28:37.513783 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.513755    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2016e952-133e-4314-a18a-40a9033a0317" (UID: "2016e952-133e-4314-a18a-40a9033a0317"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:28:37.513851 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.513804    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2016e952-133e-4314-a18a-40a9033a0317-kube-api-access-s7rc9" (OuterVolumeSpecName: "kube-api-access-s7rc9") pod "2016e952-133e-4314-a18a-40a9033a0317" (UID: "2016e952-133e-4314-a18a-40a9033a0317"). InnerVolumeSpecName "kube-api-access-s7rc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:28:37.612479 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.612440    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2016e952-133e-4314-a18a-40a9033a0317-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:28:37.612479 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.612474    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7rc9\" (UniqueName: \"kubernetes.io/projected/2016e952-133e-4314-a18a-40a9033a0317-kube-api-access-s7rc9\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:28:37.612479 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:37.612484    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2016e952-133e-4314-a18a-40a9033a0317-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:28:38.246054 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.246018    2562 generic.go:356] "Generic (PLEG): container finished" podID="2016e952-133e-4314-a18a-40a9033a0317" containerID="bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385" exitCode=0
Jul 15 20:28:38.246542 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.246061    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"
Jul 15 20:28:38.246542 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.246092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5" event={"ID":"2016e952-133e-4314-a18a-40a9033a0317","Type":"ContainerDied","Data":"bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385"}
Jul 15 20:28:38.246542 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.246144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5" event={"ID":"2016e952-133e-4314-a18a-40a9033a0317","Type":"ContainerDied","Data":"51d0ce8005be1c4b7a5ed046a1473cde9f8637894b2f965ccc37e9a8cf89d735"}
Jul 15 20:28:38.246542 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.246177    2562 scope.go:117] "RemoveContainer" containerID="bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385"
Jul 15 20:28:38.255295 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.255274    2562 scope.go:117] "RemoveContainer" containerID="bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385"
Jul 15 20:28:38.255527 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:28:38.255508    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385\": container with ID starting with bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385 not found: ID does not exist" containerID="bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385"
Jul 15 20:28:38.255566 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.255536    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385"} err="failed to get container status \"bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385\": rpc error: code = NotFound desc = could not find container \"bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385\": container with ID starting with bc053cd85a46cd6104a178bf06cf481df2436d07358c63257e2f02fbbdc5f385 not found: ID does not exist"
Jul 15 20:28:38.270926 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.270897    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"]
Jul 15 20:28:38.275322 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:38.275299    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-be7332c6-59bc8ff94c-zl4z5"]
Jul 15 20:28:39.080130 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:39.080093    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2016e952-133e-4314-a18a-40a9033a0317" path="/var/lib/kubelet/pods/2016e952-133e-4314-a18a-40a9033a0317/volumes"
Jul 15 20:28:44.238399 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.238358    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"]
Jul 15 20:28:44.239008 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.238984    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2016e952-133e-4314-a18a-40a9033a0317" containerName="maas-api"
Jul 15 20:28:44.239008 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.239009    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2016e952-133e-4314-a18a-40a9033a0317" containerName="maas-api"
Jul 15 20:28:44.239185 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.239105    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2016e952-133e-4314-a18a-40a9033a0317" containerName="maas-api"
Jul 15 20:28:44.242272 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.242246    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.247833 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.247814    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b3ba75ea-serving-cert\""
Jul 15 20:28:44.254924 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.254898    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"]
Jul 15 20:28:44.380693 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.380659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.380867 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.380721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c6v8\" (UniqueName: \"kubernetes.io/projected/25c06e19-f616-4ad2-b5d5-2233b85408a8-kube-api-access-2c6v8\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.380867 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.380765    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25c06e19-f616-4ad2-b5d5-2233b85408a8-openshift-service-ca\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.481740 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.481705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25c06e19-f616-4ad2-b5d5-2233b85408a8-openshift-service-ca\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.481935 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.481760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.481935 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.481816    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c6v8\" (UniqueName: \"kubernetes.io/projected/25c06e19-f616-4ad2-b5d5-2233b85408a8-kube-api-access-2c6v8\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.482106 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:28:44.481933    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-serving-cert: secret "maas-api-e2e-ait-b3ba75ea-serving-cert" not found
Jul 15 20:28:44.482106 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:28:44.482038    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls podName:25c06e19-f616-4ad2-b5d5-2233b85408a8 nodeName:}" failed. No retries permitted until 2026-07-15 20:28:44.982020742 +0000 UTC m=+1806.430393816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls") pod "maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk" (UID: "25c06e19-f616-4ad2-b5d5-2233b85408a8") : secret "maas-api-e2e-ait-b3ba75ea-serving-cert" not found
Jul 15 20:28:44.482476 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.482451    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25c06e19-f616-4ad2-b5d5-2233b85408a8-openshift-service-ca\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.492735 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.492674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c6v8\" (UniqueName: \"kubernetes.io/projected/25c06e19-f616-4ad2-b5d5-2233b85408a8-kube-api-access-2c6v8\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.987483 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.987440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:44.989921 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:44.989900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls\") pod \"maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:45.155792 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:45.155745    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:45.296210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:45.296183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"]
Jul 15 20:28:45.298490 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:28:45.298459    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c06e19_f616_4ad2_b5d5_2233b85408a8.slice/crio-19067c0578233679b39670667efcc1e3693e217e7ae6611e878f14085e74e659 WatchSource:0}: Error finding container 19067c0578233679b39670667efcc1e3693e217e7ae6611e878f14085e74e659: Status 404 returned error can't find the container with id 19067c0578233679b39670667efcc1e3693e217e7ae6611e878f14085e74e659
Jul 15 20:28:46.288261 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:46.288224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk" event={"ID":"25c06e19-f616-4ad2-b5d5-2233b85408a8","Type":"ContainerStarted","Data":"b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99"}
Jul 15 20:28:46.288452 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:46.288278    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk" event={"ID":"25c06e19-f616-4ad2-b5d5-2233b85408a8","Type":"ContainerStarted","Data":"19067c0578233679b39670667efcc1e3693e217e7ae6611e878f14085e74e659"}
Jul 15 20:28:46.288452 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:46.288417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:28:46.311467 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:46.311411    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk" podStartSLOduration=1.85132209 podStartE2EDuration="2.311392538s" totalImagesPullingTime="460.070448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:28:46.310080479 +0000 UTC m=+1807.758453574" watchObservedRunningTime="2026-07-15 20:28:46.311392538 +0000 UTC m=+1807.759765635"
Jul 15 20:28:52.299266 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:28:52.299240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:29:49.776444 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:49.776405    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"]
Jul 15 20:29:49.777073 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:49.776663    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk" podUID="25c06e19-f616-4ad2-b5d5-2233b85408a8" containerName="maas-api" containerID="cri-o://b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99" gracePeriod=30
Jul 15 20:29:50.025916 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.025891    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:29:50.177002 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.176905    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls\") pod \"25c06e19-f616-4ad2-b5d5-2233b85408a8\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") "
Jul 15 20:29:50.177002 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.176952    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25c06e19-f616-4ad2-b5d5-2233b85408a8-openshift-service-ca\") pod \"25c06e19-f616-4ad2-b5d5-2233b85408a8\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") "
Jul 15 20:29:50.177238 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.177019    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c6v8\" (UniqueName: \"kubernetes.io/projected/25c06e19-f616-4ad2-b5d5-2233b85408a8-kube-api-access-2c6v8\") pod \"25c06e19-f616-4ad2-b5d5-2233b85408a8\" (UID: \"25c06e19-f616-4ad2-b5d5-2233b85408a8\") "
Jul 15 20:29:50.177395 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.177372    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25c06e19-f616-4ad2-b5d5-2233b85408a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25c06e19-f616-4ad2-b5d5-2233b85408a8" (UID: "25c06e19-f616-4ad2-b5d5-2233b85408a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:29:50.179024 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.178997    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25c06e19-f616-4ad2-b5d5-2233b85408a8" (UID: "25c06e19-f616-4ad2-b5d5-2233b85408a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:29:50.179139 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.179120    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c06e19-f616-4ad2-b5d5-2233b85408a8-kube-api-access-2c6v8" (OuterVolumeSpecName: "kube-api-access-2c6v8") pod "25c06e19-f616-4ad2-b5d5-2233b85408a8" (UID: "25c06e19-f616-4ad2-b5d5-2233b85408a8"). InnerVolumeSpecName "kube-api-access-2c6v8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:29:50.278330 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.278303    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25c06e19-f616-4ad2-b5d5-2233b85408a8-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:29:50.278330 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.278327    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c6v8\" (UniqueName: \"kubernetes.io/projected/25c06e19-f616-4ad2-b5d5-2233b85408a8-kube-api-access-2c6v8\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:29:50.278482 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.278336    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25c06e19-f616-4ad2-b5d5-2233b85408a8-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:29:50.600187 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.600148    2562 generic.go:356] "Generic (PLEG): container finished" podID="25c06e19-f616-4ad2-b5d5-2233b85408a8" containerID="b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99" exitCode=0
Jul 15 20:29:50.600353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.600194    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"
Jul 15 20:29:50.600353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.600221    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk" event={"ID":"25c06e19-f616-4ad2-b5d5-2233b85408a8","Type":"ContainerDied","Data":"b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99"}
Jul 15 20:29:50.600353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.600271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk" event={"ID":"25c06e19-f616-4ad2-b5d5-2233b85408a8","Type":"ContainerDied","Data":"19067c0578233679b39670667efcc1e3693e217e7ae6611e878f14085e74e659"}
Jul 15 20:29:50.600353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.600300    2562 scope.go:117] "RemoveContainer" containerID="b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99"
Jul 15 20:29:50.609151 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.609133    2562 scope.go:117] "RemoveContainer" containerID="b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99"
Jul 15 20:29:50.609399 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:29:50.609380    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99\": container with ID starting with b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99 not found: ID does not exist" containerID="b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99"
Jul 15 20:29:50.609441 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.609411    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99"} err="failed to get container status \"b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99\": rpc error: code = NotFound desc = could not find container \"b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99\": container with ID starting with b3656dece8d7e89265f86ca986814ad128a2a1a4ee55fe6d2d079e957087ed99 not found: ID does not exist"
Jul 15 20:29:50.622652 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.622626    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"]
Jul 15 20:29:50.626413 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:50.626389    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b3ba75ea-5d6fbf5465-4ntwk"]
Jul 15 20:29:51.079296 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:29:51.079263    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25c06e19-f616-4ad2-b5d5-2233b85408a8" path="/var/lib/kubelet/pods/25c06e19-f616-4ad2-b5d5-2233b85408a8/volumes"
Jul 15 20:30:07.012955 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.012920    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"]
Jul 15 20:30:07.013359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.013322    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25c06e19-f616-4ad2-b5d5-2233b85408a8" containerName="maas-api"
Jul 15 20:30:07.013359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.013333    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="25c06e19-f616-4ad2-b5d5-2233b85408a8" containerName="maas-api"
Jul 15 20:30:07.013432 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.013421    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="25c06e19-f616-4ad2-b5d5-2233b85408a8" containerName="maas-api"
Jul 15 20:30:07.018115 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.018093    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.021399 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.021375    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6dc6ee57-serving-cert\""
Jul 15 20:30:07.033150 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.033125    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"]
Jul 15 20:30:07.130864 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.130828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9xnz\" (UniqueName: \"kubernetes.io/projected/ffcfee37-b274-4f97-9021-bbb0cea7ea68-kube-api-access-f9xnz\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.131074 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.130896    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcfee37-b274-4f97-9021-bbb0cea7ea68-openshift-service-ca\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.131074 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.130923    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.232143 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.232107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcfee37-b274-4f97-9021-bbb0cea7ea68-openshift-service-ca\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.232320 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.232154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.232320 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.232208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9xnz\" (UniqueName: \"kubernetes.io/projected/ffcfee37-b274-4f97-9021-bbb0cea7ea68-kube-api-access-f9xnz\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.232398 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:30:07.232352    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-serving-cert: secret "maas-api-e2e-derive-6dc6ee57-serving-cert" not found
Jul 15 20:30:07.232433 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:30:07.232423    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls podName:ffcfee37-b274-4f97-9021-bbb0cea7ea68 nodeName:}" failed. No retries permitted until 2026-07-15 20:30:07.732406272 +0000 UTC m=+1889.180779350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls") pod "maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl" (UID: "ffcfee37-b274-4f97-9021-bbb0cea7ea68") : secret "maas-api-e2e-derive-6dc6ee57-serving-cert" not found
Jul 15 20:30:07.232764 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.232743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcfee37-b274-4f97-9021-bbb0cea7ea68-openshift-service-ca\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.254431 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.254404    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9xnz\" (UniqueName: \"kubernetes.io/projected/ffcfee37-b274-4f97-9021-bbb0cea7ea68-kube-api-access-f9xnz\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.737458 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.737422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.739955 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.739927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls\") pod \"maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:07.927745 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:07.927675    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:08.075221 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:08.074948    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"]
Jul 15 20:30:08.077830 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:30:08.077803    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffcfee37_b274_4f97_9021_bbb0cea7ea68.slice/crio-7b4fdbf09fcca9397c8d06c735065dde1dc1f9b55b30b3f7eab62f2beb8c986b WatchSource:0}: Error finding container 7b4fdbf09fcca9397c8d06c735065dde1dc1f9b55b30b3f7eab62f2beb8c986b: Status 404 returned error can't find the container with id 7b4fdbf09fcca9397c8d06c735065dde1dc1f9b55b30b3f7eab62f2beb8c986b
Jul 15 20:30:08.695290 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:08.695252    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl" event={"ID":"ffcfee37-b274-4f97-9021-bbb0cea7ea68","Type":"ContainerStarted","Data":"7b4fdbf09fcca9397c8d06c735065dde1dc1f9b55b30b3f7eab62f2beb8c986b"}
Jul 15 20:30:09.701106 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:09.701073    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl" event={"ID":"ffcfee37-b274-4f97-9021-bbb0cea7ea68","Type":"ContainerStarted","Data":"3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c"}
Jul 15 20:30:09.701535 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:09.701137    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:30:09.726058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:09.725998    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl" podStartSLOduration=2.9588652250000003 podStartE2EDuration="3.72594565s" totalImagesPullingTime="767.080425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:30:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:30:09.725495915 +0000 UTC m=+1891.173869010" watchObservedRunningTime="2026-07-15 20:30:09.72594565 +0000 UTC m=+1891.174318748"
Jul 15 20:30:15.713712 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:30:15.713683    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:31:21.974446 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:21.974407    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"]
Jul 15 20:31:21.975005 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:21.974661    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl" podUID="ffcfee37-b274-4f97-9021-bbb0cea7ea68" containerName="maas-api" containerID="cri-o://3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c" gracePeriod=30
Jul 15 20:31:22.221426 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.221401    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:31:22.310590 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.310513    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls\") pod \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") "
Jul 15 20:31:22.310590 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.310575    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcfee37-b274-4f97-9021-bbb0cea7ea68-openshift-service-ca\") pod \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") "
Jul 15 20:31:22.310842 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.310599    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9xnz\" (UniqueName: \"kubernetes.io/projected/ffcfee37-b274-4f97-9021-bbb0cea7ea68-kube-api-access-f9xnz\") pod \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\" (UID: \"ffcfee37-b274-4f97-9021-bbb0cea7ea68\") "
Jul 15 20:31:22.311013 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.310956    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffcfee37-b274-4f97-9021-bbb0cea7ea68-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ffcfee37-b274-4f97-9021-bbb0cea7ea68" (UID: "ffcfee37-b274-4f97-9021-bbb0cea7ea68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:31:22.312631 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.312608    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ffcfee37-b274-4f97-9021-bbb0cea7ea68" (UID: "ffcfee37-b274-4f97-9021-bbb0cea7ea68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:31:22.312719 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.312635    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffcfee37-b274-4f97-9021-bbb0cea7ea68-kube-api-access-f9xnz" (OuterVolumeSpecName: "kube-api-access-f9xnz") pod "ffcfee37-b274-4f97-9021-bbb0cea7ea68" (UID: "ffcfee37-b274-4f97-9021-bbb0cea7ea68"). InnerVolumeSpecName "kube-api-access-f9xnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:31:22.411696 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.411665    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcfee37-b274-4f97-9021-bbb0cea7ea68-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:31:22.411696 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.411691    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9xnz\" (UniqueName: \"kubernetes.io/projected/ffcfee37-b274-4f97-9021-bbb0cea7ea68-kube-api-access-f9xnz\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:31:22.411696 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:22.411702    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcfee37-b274-4f97-9021-bbb0cea7ea68-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:31:23.037416 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.037380    2562 generic.go:356] "Generic (PLEG): container finished" podID="ffcfee37-b274-4f97-9021-bbb0cea7ea68" containerID="3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c" exitCode=0
Jul 15 20:31:23.037814 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.037439    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"
Jul 15 20:31:23.037814 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.037468    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl" event={"ID":"ffcfee37-b274-4f97-9021-bbb0cea7ea68","Type":"ContainerDied","Data":"3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c"}
Jul 15 20:31:23.037814 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.037554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl" event={"ID":"ffcfee37-b274-4f97-9021-bbb0cea7ea68","Type":"ContainerDied","Data":"7b4fdbf09fcca9397c8d06c735065dde1dc1f9b55b30b3f7eab62f2beb8c986b"}
Jul 15 20:31:23.037814 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.037584    2562 scope.go:117] "RemoveContainer" containerID="3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c"
Jul 15 20:31:23.046574 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.046558    2562 scope.go:117] "RemoveContainer" containerID="3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c"
Jul 15 20:31:23.046830 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:31:23.046812    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c\": container with ID starting with 3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c not found: ID does not exist" containerID="3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c"
Jul 15 20:31:23.046879 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.046841    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c"} err="failed to get container status \"3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c\": rpc error: code = NotFound desc = could not find container \"3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c\": container with ID starting with 3de79dcab403f2cd07d1db078191641a5f963e91eff8db65dc8e658b982fee1c not found: ID does not exist"
Jul 15 20:31:23.060443 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.060421    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"]
Jul 15 20:31:23.064583 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.064564    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6dc6ee57-545bb5d447-5wdbl"]
Jul 15 20:31:23.079063 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:31:23.079036    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffcfee37-b274-4f97-9021-bbb0cea7ea68" path="/var/lib/kubelet/pods/ffcfee37-b274-4f97-9021-bbb0cea7ea68/volumes"
Jul 15 20:33:16.870629 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:16.870550    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"]
Jul 15 20:33:16.871142 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:16.870953    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffcfee37-b274-4f97-9021-bbb0cea7ea68" containerName="maas-api"
Jul 15 20:33:16.871142 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:16.870978    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffcfee37-b274-4f97-9021-bbb0cea7ea68" containerName="maas-api"
Jul 15 20:33:16.871142 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:16.871048    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffcfee37-b274-4f97-9021-bbb0cea7ea68" containerName="maas-api"
Jul 15 20:33:16.873946 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:16.873929    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:16.879867 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:16.879846    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-cea3e2bc-serving-cert\""
Jul 15 20:33:16.897814 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:16.897783    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"]
Jul 15 20:33:17.030241 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.030199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e403f990-8154-4dc1-a425-15607df4c28e-maas-api-tls\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.030241 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.030241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffz4q\" (UniqueName: \"kubernetes.io/projected/e403f990-8154-4dc1-a425-15607df4c28e-kube-api-access-ffz4q\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.030445 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.030314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e403f990-8154-4dc1-a425-15607df4c28e-openshift-service-ca\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.131543 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.131457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e403f990-8154-4dc1-a425-15607df4c28e-openshift-service-ca\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.131691 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.131630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e403f990-8154-4dc1-a425-15607df4c28e-maas-api-tls\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.131691 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.131662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffz4q\" (UniqueName: \"kubernetes.io/projected/e403f990-8154-4dc1-a425-15607df4c28e-kube-api-access-ffz4q\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.132131 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.132109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e403f990-8154-4dc1-a425-15607df4c28e-openshift-service-ca\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.134099 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.134074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e403f990-8154-4dc1-a425-15607df4c28e-maas-api-tls\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.156774 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.156746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffz4q\" (UniqueName: \"kubernetes.io/projected/e403f990-8154-4dc1-a425-15607df4c28e-kube-api-access-ffz4q\") pod \"maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.183895 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.183860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:17.329862 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.329837    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"]
Jul 15 20:33:17.331781 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:33:17.331750    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode403f990_8154_4dc1_a425_15607df4c28e.slice/crio-8a47c89c563c4c198753a258be775036da28d4bb08a10d7614cf1833f3b627b9 WatchSource:0}: Error finding container 8a47c89c563c4c198753a258be775036da28d4bb08a10d7614cf1833f3b627b9: Status 404 returned error can't find the container with id 8a47c89c563c4c198753a258be775036da28d4bb08a10d7614cf1833f3b627b9
Jul 15 20:33:17.333280 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.333260    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:33:17.563035 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:17.562999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9" event={"ID":"e403f990-8154-4dc1-a425-15607df4c28e","Type":"ContainerStarted","Data":"8a47c89c563c4c198753a258be775036da28d4bb08a10d7614cf1833f3b627b9"}
Jul 15 20:33:18.569325 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:18.569289    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9" event={"ID":"e403f990-8154-4dc1-a425-15607df4c28e","Type":"ContainerStarted","Data":"d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983"}
Jul 15 20:33:18.569739 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:18.569408    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:18.589240 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:18.589188    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9" podStartSLOduration=2.083822857 podStartE2EDuration="2.589170258s" totalImagesPullingTime="505.347401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:33:18.587998151 +0000 UTC m=+2080.036371247" watchObservedRunningTime="2026-07-15 20:33:18.589170258 +0000 UTC m=+2080.037543354"
Jul 15 20:33:24.581370 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:24.581334    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:38.922433 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:38.922399    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"]
Jul 15 20:33:38.922920 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:38.922688    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9" podUID="e403f990-8154-4dc1-a425-15607df4c28e" containerName="maas-api" containerID="cri-o://d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983" gracePeriod=30
Jul 15 20:33:39.169278 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.169253    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:39.212624 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.212598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffz4q\" (UniqueName: \"kubernetes.io/projected/e403f990-8154-4dc1-a425-15607df4c28e-kube-api-access-ffz4q\") pod \"e403f990-8154-4dc1-a425-15607df4c28e\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") "
Jul 15 20:33:39.212797 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.212711    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e403f990-8154-4dc1-a425-15607df4c28e-maas-api-tls\") pod \"e403f990-8154-4dc1-a425-15607df4c28e\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") "
Jul 15 20:33:39.212797 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.212744    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e403f990-8154-4dc1-a425-15607df4c28e-openshift-service-ca\") pod \"e403f990-8154-4dc1-a425-15607df4c28e\" (UID: \"e403f990-8154-4dc1-a425-15607df4c28e\") "
Jul 15 20:33:39.213189 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.213159    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e403f990-8154-4dc1-a425-15607df4c28e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e403f990-8154-4dc1-a425-15607df4c28e" (UID: "e403f990-8154-4dc1-a425-15607df4c28e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:33:39.214643 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.214616    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e403f990-8154-4dc1-a425-15607df4c28e-kube-api-access-ffz4q" (OuterVolumeSpecName: "kube-api-access-ffz4q") pod "e403f990-8154-4dc1-a425-15607df4c28e" (UID: "e403f990-8154-4dc1-a425-15607df4c28e"). InnerVolumeSpecName "kube-api-access-ffz4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:33:39.214756 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.214644    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e403f990-8154-4dc1-a425-15607df4c28e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e403f990-8154-4dc1-a425-15607df4c28e" (UID: "e403f990-8154-4dc1-a425-15607df4c28e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:33:39.313705 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.313676    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffz4q\" (UniqueName: \"kubernetes.io/projected/e403f990-8154-4dc1-a425-15607df4c28e-kube-api-access-ffz4q\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:33:39.313705 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.313701    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e403f990-8154-4dc1-a425-15607df4c28e-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:33:39.313705 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.313710    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e403f990-8154-4dc1-a425-15607df4c28e-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:33:39.666854 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.666763    2562 generic.go:356] "Generic (PLEG): container finished" podID="e403f990-8154-4dc1-a425-15607df4c28e" containerID="d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983" exitCode=0
Jul 15 20:33:39.666854 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.666804    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"
Jul 15 20:33:39.666854 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.666811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9" event={"ID":"e403f990-8154-4dc1-a425-15607df4c28e","Type":"ContainerDied","Data":"d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983"}
Jul 15 20:33:39.667157 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.666862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9" event={"ID":"e403f990-8154-4dc1-a425-15607df4c28e","Type":"ContainerDied","Data":"8a47c89c563c4c198753a258be775036da28d4bb08a10d7614cf1833f3b627b9"}
Jul 15 20:33:39.667157 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.666891    2562 scope.go:117] "RemoveContainer" containerID="d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983"
Jul 15 20:33:39.678184 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.676797    2562 scope.go:117] "RemoveContainer" containerID="d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983"
Jul 15 20:33:39.681073 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:33:39.681046    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983\": container with ID starting with d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983 not found: ID does not exist" containerID="d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983"
Jul 15 20:33:39.681194 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.681079    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983"} err="failed to get container status \"d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983\": rpc error: code = NotFound desc = could not find container \"d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983\": container with ID starting with d263e410fa6fa1dcba96de8f11c3c794475d85f679518106953a5f4323f4b983 not found: ID does not exist"
Jul 15 20:33:39.692626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.692600    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"]
Jul 15 20:33:39.695172 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:39.695151    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cea3e2bc-75c99bd96d-hm9x9"]
Jul 15 20:33:41.079164 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:33:41.079130    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e403f990-8154-4dc1-a425-15607df4c28e" path="/var/lib/kubelet/pods/e403f990-8154-4dc1-a425-15607df4c28e/volumes"
Jul 15 20:34:17.159989 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.159691    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"]
Jul 15 20:34:17.160873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.160389    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e403f990-8154-4dc1-a425-15607df4c28e" containerName="maas-api"
Jul 15 20:34:17.160873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.160422    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e403f990-8154-4dc1-a425-15607df4c28e" containerName="maas-api"
Jul 15 20:34:17.160873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.160521    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e403f990-8154-4dc1-a425-15607df4c28e" containerName="maas-api"
Jul 15 20:34:17.177996 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.176459    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.192015 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.191027    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2bf8d569-serving-cert\""
Jul 15 20:34:17.196724 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.196697    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"]
Jul 15 20:34:17.257895 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.257864    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2f47a3-72f8-4b54-933e-b000955b103e-maas-api-tls\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.257895 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.257898    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpw96\" (UniqueName: \"kubernetes.io/projected/fe2f47a3-72f8-4b54-933e-b000955b103e-kube-api-access-rpw96\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.258162 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.258024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2f47a3-72f8-4b54-933e-b000955b103e-openshift-service-ca\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.359058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.359025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2f47a3-72f8-4b54-933e-b000955b103e-openshift-service-ca\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.359236 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.359132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2f47a3-72f8-4b54-933e-b000955b103e-maas-api-tls\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.359236 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.359149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpw96\" (UniqueName: \"kubernetes.io/projected/fe2f47a3-72f8-4b54-933e-b000955b103e-kube-api-access-rpw96\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.359698 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.359673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2f47a3-72f8-4b54-933e-b000955b103e-openshift-service-ca\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.361476 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.361456    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2f47a3-72f8-4b54-933e-b000955b103e-maas-api-tls\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.371130 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.371106    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpw96\" (UniqueName: \"kubernetes.io/projected/fe2f47a3-72f8-4b54-933e-b000955b103e-kube-api-access-rpw96\") pod \"maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.499620 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.499581    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:17.852910 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:17.852885    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"]
Jul 15 20:34:17.855348 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:34:17.855314    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2f47a3_72f8_4b54_933e_b000955b103e.slice/crio-ee4cf229b5a0a8ba9171e8c7b04c34b8c15265e77b7fa6e6f3ab6c5be839e7d6 WatchSource:0}: Error finding container ee4cf229b5a0a8ba9171e8c7b04c34b8c15265e77b7fa6e6f3ab6c5be839e7d6: Status 404 returned error can't find the container with id ee4cf229b5a0a8ba9171e8c7b04c34b8c15265e77b7fa6e6f3ab6c5be839e7d6
Jul 15 20:34:18.849356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:18.849316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5" event={"ID":"fe2f47a3-72f8-4b54-933e-b000955b103e","Type":"ContainerStarted","Data":"41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d"}
Jul 15 20:34:18.849721 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:18.849367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5" event={"ID":"fe2f47a3-72f8-4b54-933e-b000955b103e","Type":"ContainerStarted","Data":"ee4cf229b5a0a8ba9171e8c7b04c34b8c15265e77b7fa6e6f3ab6c5be839e7d6"}
Jul 15 20:34:18.849721 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:18.849463    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:18.869044 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:18.868996    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5" podStartSLOduration=1.417013702 podStartE2EDuration="1.868963718s" totalImagesPullingTime="451.950016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:34:18.867714887 +0000 UTC m=+2140.316087982" watchObservedRunningTime="2026-07-15 20:34:18.868963718 +0000 UTC m=+2140.317336814"
Jul 15 20:34:24.861355 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:24.861319    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:34:30.973430 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:30.973352    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"]
Jul 15 20:34:30.977058 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:30.977036    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:30.981092 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:30.981067    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2bf8d569\"/\"kube-root-ca.crt\""
Jul 15 20:34:30.982456 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:30.982434    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2bf8d569\"/\"e2e-lifecycle-model-2bf8d569-kserve-self-signed-certs\""
Jul 15 20:34:30.982456 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:30.982454    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2bf8d569\"/\"default-dockercfg-22xk6\""
Jul 15 20:34:30.982622 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:30.982485    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2bf8d569\"/\"openshift-service-ca.crt\""
Jul 15 20:34:30.992928 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:30.992907    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"]
Jul 15 20:34:31.080102 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.080069    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-tmp-dir\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.080102 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.080106    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-model-cache\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.080301 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.080128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34ba780b-a8e0-4220-8aa8-3864de2e9615-tls-certs\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.080301 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.080201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-dshm\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.080301 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.080244    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-home\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.080301 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.080265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glj7p\" (UniqueName: \"kubernetes.io/projected/34ba780b-a8e0-4220-8aa8-3864de2e9615-kube-api-access-glj7p\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.180844 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.180808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-tmp-dir\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.180844 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.180847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-model-cache\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.181075 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.180870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34ba780b-a8e0-4220-8aa8-3864de2e9615-tls-certs\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.181075 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.180900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-dshm\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.181075 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.180919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-home\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.181075 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.180937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glj7p\" (UniqueName: \"kubernetes.io/projected/34ba780b-a8e0-4220-8aa8-3864de2e9615-kube-api-access-glj7p\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.181321 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.181298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-tmp-dir\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.181378 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.181308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-model-cache\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.181427 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.181382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-home\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.183163 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.183144    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-dshm\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.183459 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.183441    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34ba780b-a8e0-4220-8aa8-3864de2e9615-tls-certs\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.189627 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.189604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glj7p\" (UniqueName: \"kubernetes.io/projected/34ba780b-a8e0-4220-8aa8-3864de2e9615-kube-api-access-glj7p\") pod \"e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") " pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.296317 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.296254    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:31.428525 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.428506    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"]
Jul 15 20:34:31.431009 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:34:31.430982    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34ba780b_a8e0_4220_8aa8_3864de2e9615.slice/crio-2c307dd81bf80a3cfb84c9bd54bc51961a40e94cd3c8e566c6af705fc5fa20ef WatchSource:0}: Error finding container 2c307dd81bf80a3cfb84c9bd54bc51961a40e94cd3c8e566c6af705fc5fa20ef: Status 404 returned error can't find the container with id 2c307dd81bf80a3cfb84c9bd54bc51961a40e94cd3c8e566c6af705fc5fa20ef
Jul 15 20:34:31.912627 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.912597    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m" event={"ID":"34ba780b-a8e0-4220-8aa8-3864de2e9615","Type":"ContainerStarted","Data":"2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6"}
Jul 15 20:34:31.912805 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.912637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m" event={"ID":"34ba780b-a8e0-4220-8aa8-3864de2e9615","Type":"ContainerStarted","Data":"2c307dd81bf80a3cfb84c9bd54bc51961a40e94cd3c8e566c6af705fc5fa20ef"}
Jul 15 20:34:31.932030 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:31.931982    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m" podStartSLOduration=1.931954024 podStartE2EDuration="1.931954024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:34:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:34:31.930183018 +0000 UTC m=+2153.378556118" watchObservedRunningTime="2026-07-15 20:34:31.931954024 +0000 UTC m=+2153.380327119"
Jul 15 20:34:41.296783 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:41.296747    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:41.297216 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:41.296794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:41.309370 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:41.309347    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:34:41.971778 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:34:41.971753    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:35:05.125149 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.125114    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"]
Jul 15 20:35:05.125750 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.125318    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5" podUID="fe2f47a3-72f8-4b54-933e-b000955b103e" containerName="maas-api" containerID="cri-o://41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d" gracePeriod=30
Jul 15 20:35:05.374740 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.374713    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:35:05.392359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.392271    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2f47a3-72f8-4b54-933e-b000955b103e-maas-api-tls\") pod \"fe2f47a3-72f8-4b54-933e-b000955b103e\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") "
Jul 15 20:35:05.392359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.392308    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpw96\" (UniqueName: \"kubernetes.io/projected/fe2f47a3-72f8-4b54-933e-b000955b103e-kube-api-access-rpw96\") pod \"fe2f47a3-72f8-4b54-933e-b000955b103e\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") "
Jul 15 20:35:05.392359 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.392343    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2f47a3-72f8-4b54-933e-b000955b103e-openshift-service-ca\") pod \"fe2f47a3-72f8-4b54-933e-b000955b103e\" (UID: \"fe2f47a3-72f8-4b54-933e-b000955b103e\") "
Jul 15 20:35:05.392846 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.392795    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe2f47a3-72f8-4b54-933e-b000955b103e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe2f47a3-72f8-4b54-933e-b000955b103e" (UID: "fe2f47a3-72f8-4b54-933e-b000955b103e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:35:05.395004 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.394951    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe2f47a3-72f8-4b54-933e-b000955b103e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe2f47a3-72f8-4b54-933e-b000955b103e" (UID: "fe2f47a3-72f8-4b54-933e-b000955b103e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:35:05.395269 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.395239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe2f47a3-72f8-4b54-933e-b000955b103e-kube-api-access-rpw96" (OuterVolumeSpecName: "kube-api-access-rpw96") pod "fe2f47a3-72f8-4b54-933e-b000955b103e" (UID: "fe2f47a3-72f8-4b54-933e-b000955b103e"). InnerVolumeSpecName "kube-api-access-rpw96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:35:05.493315 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.493283    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2f47a3-72f8-4b54-933e-b000955b103e-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:05.493315 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.493309    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpw96\" (UniqueName: \"kubernetes.io/projected/fe2f47a3-72f8-4b54-933e-b000955b103e-kube-api-access-rpw96\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:05.493315 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:05.493318    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2f47a3-72f8-4b54-933e-b000955b103e-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:06.070091 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.070056    2562 generic.go:356] "Generic (PLEG): container finished" podID="fe2f47a3-72f8-4b54-933e-b000955b103e" containerID="41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d" exitCode=0
Jul 15 20:35:06.070289 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.070098    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"
Jul 15 20:35:06.070289 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.070142    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5" event={"ID":"fe2f47a3-72f8-4b54-933e-b000955b103e","Type":"ContainerDied","Data":"41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d"}
Jul 15 20:35:06.070289 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.070197    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5" event={"ID":"fe2f47a3-72f8-4b54-933e-b000955b103e","Type":"ContainerDied","Data":"ee4cf229b5a0a8ba9171e8c7b04c34b8c15265e77b7fa6e6f3ab6c5be839e7d6"}
Jul 15 20:35:06.070289 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.070225    2562 scope.go:117] "RemoveContainer" containerID="41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d"
Jul 15 20:35:06.079267 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.079249    2562 scope.go:117] "RemoveContainer" containerID="41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d"
Jul 15 20:35:06.079553 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:35:06.079534    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d\": container with ID starting with 41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d not found: ID does not exist" containerID="41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d"
Jul 15 20:35:06.079601 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.079563    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d"} err="failed to get container status \"41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d\": rpc error: code = NotFound desc = could not find container \"41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d\": container with ID starting with 41ed1df288c8b95c05c95410efb353501ec9338d85c97bdf95ae56a7c94b0d3d not found: ID does not exist"
Jul 15 20:35:06.092953 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.092928    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"]
Jul 15 20:35:06.096135 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:06.096114    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2bf8d569-5b8479dff6-vhmb5"]
Jul 15 20:35:07.077863 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:07.077833    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe2f47a3-72f8-4b54-933e-b000955b103e" path="/var/lib/kubelet/pods/fe2f47a3-72f8-4b54-933e-b000955b103e/volumes"
Jul 15 20:35:14.744178 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:14.744140    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"]
Jul 15 20:35:29.765729 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:29.765663    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m" podUID="34ba780b-a8e0-4220-8aa8-3864de2e9615" containerName="main" containerID="cri-o://2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6" gracePeriod=45
Jul 15 20:35:30.021645 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.021580    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:35:30.125559 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125524    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-tmp-dir\") pod \"34ba780b-a8e0-4220-8aa8-3864de2e9615\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") "
Jul 15 20:35:30.125559 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125559    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glj7p\" (UniqueName: \"kubernetes.io/projected/34ba780b-a8e0-4220-8aa8-3864de2e9615-kube-api-access-glj7p\") pod \"34ba780b-a8e0-4220-8aa8-3864de2e9615\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") "
Jul 15 20:35:30.125797 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125604    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-home\") pod \"34ba780b-a8e0-4220-8aa8-3864de2e9615\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") "
Jul 15 20:35:30.125797 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34ba780b-a8e0-4220-8aa8-3864de2e9615-tls-certs\") pod \"34ba780b-a8e0-4220-8aa8-3864de2e9615\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") "
Jul 15 20:35:30.125797 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125670    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-dshm\") pod \"34ba780b-a8e0-4220-8aa8-3864de2e9615\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") "
Jul 15 20:35:30.125797 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125725    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-model-cache\") pod \"34ba780b-a8e0-4220-8aa8-3864de2e9615\" (UID: \"34ba780b-a8e0-4220-8aa8-3864de2e9615\") "
Jul 15 20:35:30.126034 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125807    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "34ba780b-a8e0-4220-8aa8-3864de2e9615" (UID: "34ba780b-a8e0-4220-8aa8-3864de2e9615"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:35:30.126034 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.125906    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-home" (OuterVolumeSpecName: "home") pod "34ba780b-a8e0-4220-8aa8-3864de2e9615" (UID: "34ba780b-a8e0-4220-8aa8-3864de2e9615"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:35:30.126135 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.126061    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:30.126173 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.126125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-model-cache" (OuterVolumeSpecName: "model-cache") pod "34ba780b-a8e0-4220-8aa8-3864de2e9615" (UID: "34ba780b-a8e0-4220-8aa8-3864de2e9615"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:35:30.127879 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.127843    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34ba780b-a8e0-4220-8aa8-3864de2e9615-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "34ba780b-a8e0-4220-8aa8-3864de2e9615" (UID: "34ba780b-a8e0-4220-8aa8-3864de2e9615"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:35:30.127879 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.127846    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-dshm" (OuterVolumeSpecName: "dshm") pod "34ba780b-a8e0-4220-8aa8-3864de2e9615" (UID: "34ba780b-a8e0-4220-8aa8-3864de2e9615"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:35:30.128059 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.127889    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34ba780b-a8e0-4220-8aa8-3864de2e9615-kube-api-access-glj7p" (OuterVolumeSpecName: "kube-api-access-glj7p") pod "34ba780b-a8e0-4220-8aa8-3864de2e9615" (UID: "34ba780b-a8e0-4220-8aa8-3864de2e9615"). InnerVolumeSpecName "kube-api-access-glj7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:35:30.183420 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.183385    2562 generic.go:356] "Generic (PLEG): container finished" podID="34ba780b-a8e0-4220-8aa8-3864de2e9615" containerID="2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6" exitCode=0
Jul 15 20:35:30.183548 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.183434    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"
Jul 15 20:35:30.183548 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.183467    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m" event={"ID":"34ba780b-a8e0-4220-8aa8-3864de2e9615","Type":"ContainerDied","Data":"2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6"}
Jul 15 20:35:30.183548 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.183510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m" event={"ID":"34ba780b-a8e0-4220-8aa8-3864de2e9615","Type":"ContainerDied","Data":"2c307dd81bf80a3cfb84c9bd54bc51961a40e94cd3c8e566c6af705fc5fa20ef"}
Jul 15 20:35:30.183548 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.183527    2562 scope.go:117] "RemoveContainer" containerID="2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6"
Jul 15 20:35:30.192342 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.192326    2562 scope.go:117] "RemoveContainer" containerID="2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6"
Jul 15 20:35:30.192588 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:35:30.192570    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6\": container with ID starting with 2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6 not found: ID does not exist" containerID="2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6"
Jul 15 20:35:30.192649 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.192600    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6"} err="failed to get container status \"2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6\": rpc error: code = NotFound desc = could not find container \"2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6\": container with ID starting with 2df40ba42ae7c0a01b8c60315be58b11250f3fac362906beaa8ac699364e82f6 not found: ID does not exist"
Jul 15 20:35:30.207573 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.207545    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"]
Jul 15 20:35:30.209178 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.209159    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2bf8d569/e2e-lifecycle-model-2bf8d569-kserve-cc6998c5-4l28m"]
Jul 15 20:35:30.227175 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.227149    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:30.227263 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.227177    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glj7p\" (UniqueName: \"kubernetes.io/projected/34ba780b-a8e0-4220-8aa8-3864de2e9615-kube-api-access-glj7p\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:30.227263 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.227191    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:30.227263 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.227207    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34ba780b-a8e0-4220-8aa8-3864de2e9615-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:30.227263 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:30.227218    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34ba780b-a8e0-4220-8aa8-3864de2e9615-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:35:31.078886 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:35:31.078844    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34ba780b-a8e0-4220-8aa8-3864de2e9615" path="/var/lib/kubelet/pods/34ba780b-a8e0-4220-8aa8-3864de2e9615/volumes"
Jul 15 20:36:59.915418 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.915387    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"]
Jul 15 20:36:59.916027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.915753    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe2f47a3-72f8-4b54-933e-b000955b103e" containerName="maas-api"
Jul 15 20:36:59.916027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.915764    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe2f47a3-72f8-4b54-933e-b000955b103e" containerName="maas-api"
Jul 15 20:36:59.916027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.915779    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ba780b-a8e0-4220-8aa8-3864de2e9615" containerName="main"
Jul 15 20:36:59.916027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.915785    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ba780b-a8e0-4220-8aa8-3864de2e9615" containerName="main"
Jul 15 20:36:59.916027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.915849    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ba780b-a8e0-4220-8aa8-3864de2e9615" containerName="main"
Jul 15 20:36:59.916027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.915858    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe2f47a3-72f8-4b54-933e-b000955b103e" containerName="maas-api"
Jul 15 20:36:59.919032 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.919010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:36:59.921324 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.921301    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b06d27-serving-cert\""
Jul 15 20:36:59.929369 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:36:59.929346    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"]
Jul 15 20:37:00.019875 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.019842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16d826d9-b178-403a-93be-aadda86ed36f-maas-api-tls\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.019875 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.019876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d826d9-b178-403a-93be-aadda86ed36f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.020125 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.020022    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cqvh\" (UniqueName: \"kubernetes.io/projected/16d826d9-b178-403a-93be-aadda86ed36f-kube-api-access-8cqvh\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.120906 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.120873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqvh\" (UniqueName: \"kubernetes.io/projected/16d826d9-b178-403a-93be-aadda86ed36f-kube-api-access-8cqvh\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.121094 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.121008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16d826d9-b178-403a-93be-aadda86ed36f-maas-api-tls\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.121094 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.121044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d826d9-b178-403a-93be-aadda86ed36f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.121712 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.121690    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d826d9-b178-403a-93be-aadda86ed36f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.123417 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.123395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16d826d9-b178-403a-93be-aadda86ed36f-maas-api-tls\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.129320 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.129298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cqvh\" (UniqueName: \"kubernetes.io/projected/16d826d9-b178-403a-93be-aadda86ed36f-kube-api-access-8cqvh\") pod \"maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.230708 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.230678    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:00.364825 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.364793    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"]
Jul 15 20:37:00.366467 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:37:00.366440    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d826d9_b178_403a_93be_aadda86ed36f.slice/crio-a3f98a519be88e1dfb0a2a61c21785ec94c024c8e91e77d7c56fbf67cacee051 WatchSource:0}: Error finding container a3f98a519be88e1dfb0a2a61c21785ec94c024c8e91e77d7c56fbf67cacee051: Status 404 returned error can't find the container with id a3f98a519be88e1dfb0a2a61c21785ec94c024c8e91e77d7c56fbf67cacee051
Jul 15 20:37:00.603593 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:00.603507    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv" event={"ID":"16d826d9-b178-403a-93be-aadda86ed36f","Type":"ContainerStarted","Data":"a3f98a519be88e1dfb0a2a61c21785ec94c024c8e91e77d7c56fbf67cacee051"}
Jul 15 20:37:01.609051 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:01.609013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv" event={"ID":"16d826d9-b178-403a-93be-aadda86ed36f","Type":"ContainerStarted","Data":"f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1"}
Jul 15 20:37:01.609505 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:01.609139    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:01.627720 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:01.627671    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv" podStartSLOduration=2.119252084 podStartE2EDuration="2.627656276s" totalImagesPullingTime="508.404192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:37:01.626651467 +0000 UTC m=+2303.075024563" watchObservedRunningTime="2026-07-15 20:37:01.627656276 +0000 UTC m=+2303.076029372"
Jul 15 20:37:07.618840 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:07.618806    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:37:07.941391 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:07.941348    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"]
Jul 15 20:37:07.945210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:07.945189    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:07.947699 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:07.947679    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e45884-serving-cert\""
Jul 15 20:37:07.955541 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:07.955519    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"]
Jul 15 20:37:08.089867 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.089826    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdm2v\" (UniqueName: \"kubernetes.io/projected/9ed31653-94fc-4b4d-8480-72b08b1b788d-kube-api-access-fdm2v\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.090096 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.089884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed31653-94fc-4b4d-8480-72b08b1b788d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.090096 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.089926    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed31653-94fc-4b4d-8480-72b08b1b788d-maas-api-tls\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.191484 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.191393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdm2v\" (UniqueName: \"kubernetes.io/projected/9ed31653-94fc-4b4d-8480-72b08b1b788d-kube-api-access-fdm2v\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.191484 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.191448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed31653-94fc-4b4d-8480-72b08b1b788d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.191484 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.191476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed31653-94fc-4b4d-8480-72b08b1b788d-maas-api-tls\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.192261 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.192232    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed31653-94fc-4b4d-8480-72b08b1b788d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.193880 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.193858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed31653-94fc-4b4d-8480-72b08b1b788d-maas-api-tls\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.205725 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.205705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdm2v\" (UniqueName: \"kubernetes.io/projected/9ed31653-94fc-4b4d-8480-72b08b1b788d-kube-api-access-fdm2v\") pod \"maas-api-e2e-inf-b-e45884-5448d69559-vkfxl\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.256855 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.256804    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:08.397209 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.397186    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"]
Jul 15 20:37:08.399498 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:37:08.399471    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed31653_94fc_4b4d_8480_72b08b1b788d.slice/crio-e94c2e7ecc27711928d146ba9a6b8dd8b27e34d2e3fd166671ca0f90be2754a4 WatchSource:0}: Error finding container e94c2e7ecc27711928d146ba9a6b8dd8b27e34d2e3fd166671ca0f90be2754a4: Status 404 returned error can't find the container with id e94c2e7ecc27711928d146ba9a6b8dd8b27e34d2e3fd166671ca0f90be2754a4
Jul 15 20:37:08.643429 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:08.643339    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl" event={"ID":"9ed31653-94fc-4b4d-8480-72b08b1b788d","Type":"ContainerStarted","Data":"e94c2e7ecc27711928d146ba9a6b8dd8b27e34d2e3fd166671ca0f90be2754a4"}
Jul 15 20:37:09.649261 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:09.649220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl" event={"ID":"9ed31653-94fc-4b4d-8480-72b08b1b788d","Type":"ContainerStarted","Data":"c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218"}
Jul 15 20:37:09.649681 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:09.649385    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:09.668800 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:09.668749    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl" podStartSLOduration=2.073819805 podStartE2EDuration="2.6687306s" totalImagesPullingTime="594.910795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:37:09.665898091 +0000 UTC m=+2311.114271186" watchObservedRunningTime="2026-07-15 20:37:09.6687306 +0000 UTC m=+2311.117103696"
Jul 15 20:37:15.659229 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:15.659199    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:37:21.053556 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.053519    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"]
Jul 15 20:37:21.056859 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.056842    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.061057 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.061032    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b06d27\"/\"kube-root-ca.crt\""
Jul 15 20:37:21.061201 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.061035    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b06d27\"/\"default-dockercfg-ng2m2\""
Jul 15 20:37:21.061201 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.061069    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b06d27\"/\"test-model-b06d27-kserve-self-signed-certs\""
Jul 15 20:37:21.061201 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.061089    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b06d27\"/\"openshift-service-ca.crt\""
Jul 15 20:37:21.066944 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.066919    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"]
Jul 15 20:37:21.217150 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.217116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-home\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.217355 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.217155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tmp-dir\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.217355 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.217192    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-dshm\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.217355 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.217220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmltm\" (UniqueName: \"kubernetes.io/projected/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-kube-api-access-jmltm\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.217355 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.217255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-model-cache\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.217550 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.217411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tls-certs\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.318611 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.318508    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmltm\" (UniqueName: \"kubernetes.io/projected/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-kube-api-access-jmltm\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.318789 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.318665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-model-cache\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.318789 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.318765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tls-certs\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.318906 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.318814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-home\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.318906 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.318834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tmp-dir\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.318906 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.318865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-dshm\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.319189 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.319168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-model-cache\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.319262 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.319233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tmp-dir\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.319321 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.319271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-home\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.321191 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.321170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-dshm\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.321382 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.321363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tls-certs\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.326274 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.326250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmltm\" (UniqueName: \"kubernetes.io/projected/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-kube-api-access-jmltm\") pod \"test-model-b06d27-kserve-74bccc6487-7jz2t\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") " pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.379203 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.379160    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:21.519669 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.519645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"]
Jul 15 20:37:21.521715 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:37:21.521682    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98b9897f_7b81_4bfc_b1d0_7c58618eb9d6.slice/crio-f93de0e30e89db48dffe34ea65a92ed0c1dde5656f92d19105b76ae1391e33c1 WatchSource:0}: Error finding container f93de0e30e89db48dffe34ea65a92ed0c1dde5656f92d19105b76ae1391e33c1: Status 404 returned error can't find the container with id f93de0e30e89db48dffe34ea65a92ed0c1dde5656f92d19105b76ae1391e33c1
Jul 15 20:37:21.707400 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.707356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t" event={"ID":"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6","Type":"ContainerStarted","Data":"68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a"}
Jul 15 20:37:21.707578 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.707419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t" event={"ID":"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6","Type":"ContainerStarted","Data":"f93de0e30e89db48dffe34ea65a92ed0c1dde5656f92d19105b76ae1391e33c1"}
Jul 15 20:37:21.725548 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:21.725489    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t" podStartSLOduration=0.725469978 podStartE2EDuration="725.469978ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:37:21.723511731 +0000 UTC m=+2323.171884860" watchObservedRunningTime="2026-07-15 20:37:21.725469978 +0000 UTC m=+2323.173843095"
Jul 15 20:37:31.380111 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:31.380027    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:31.380111 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:31.380087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:31.392626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:31.392602    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:31.764717 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:31.764688    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:37:38.343139 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.343107    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"]
Jul 15 20:37:38.346814 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.346795    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.350768 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.350745    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e45884\"/\"test-model-e45884-kserve-self-signed-certs\""
Jul 15 20:37:38.350897 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.350771    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e45884\"/\"default-dockercfg-hlg9p\""
Jul 15 20:37:38.350897 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.350788    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e45884\"/\"openshift-service-ca.crt\""
Jul 15 20:37:38.350897 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.350772    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e45884\"/\"kube-root-ca.crt\""
Jul 15 20:37:38.356751 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.356723    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"]
Jul 15 20:37:38.367646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.367622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlnnv\" (UniqueName: \"kubernetes.io/projected/1349e96f-c64c-4744-aac3-ef20f0b27389-kube-api-access-xlnnv\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.367761 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.367707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1349e96f-c64c-4744-aac3-ef20f0b27389-tls-certs\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.367806 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.367781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-model-cache\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.367846 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.367809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-home\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.367846 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.367842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-tmp-dir\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.367960 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.367943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-dshm\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.468774 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.468746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-tmp-dir\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.468930 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.468804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-dshm\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.468930 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.468825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlnnv\" (UniqueName: \"kubernetes.io/projected/1349e96f-c64c-4744-aac3-ef20f0b27389-kube-api-access-xlnnv\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.468930 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.468865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1349e96f-c64c-4744-aac3-ef20f0b27389-tls-certs\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.468930 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.468892    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-model-cache\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.468930 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.468909    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-home\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.469257 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.469238    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-tmp-dir\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.469353 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.469335    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-model-cache\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.469402 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.469347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-home\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.471153 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.471130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-dshm\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.471331 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.471311    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1349e96f-c64c-4744-aac3-ef20f0b27389-tls-certs\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.478207 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.478186    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlnnv\" (UniqueName: \"kubernetes.io/projected/1349e96f-c64c-4744-aac3-ef20f0b27389-kube-api-access-xlnnv\") pod \"test-model-e45884-kserve-745544b685-59sc2\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") " pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.656666 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.656589    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:38.793377 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:38.793352    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"]
Jul 15 20:37:38.795737 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:37:38.795698    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1349e96f_c64c_4744_aac3_ef20f0b27389.slice/crio-4c2ed1bcc8ccc8201e507cc9e04c5c958d68811739062a54a66e05bd3162978d WatchSource:0}: Error finding container 4c2ed1bcc8ccc8201e507cc9e04c5c958d68811739062a54a66e05bd3162978d: Status 404 returned error can't find the container with id 4c2ed1bcc8ccc8201e507cc9e04c5c958d68811739062a54a66e05bd3162978d
Jul 15 20:37:39.793273 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:39.793232    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2" event={"ID":"1349e96f-c64c-4744-aac3-ef20f0b27389","Type":"ContainerStarted","Data":"f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72"}
Jul 15 20:37:39.793698 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:39.793283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2" event={"ID":"1349e96f-c64c-4744-aac3-ef20f0b27389","Type":"ContainerStarted","Data":"4c2ed1bcc8ccc8201e507cc9e04c5c958d68811739062a54a66e05bd3162978d"}
Jul 15 20:37:39.810039 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:39.809958    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2" podStartSLOduration=1.80993911 podStartE2EDuration="1.80993911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:37:39.809340697 +0000 UTC m=+2341.257713794" watchObservedRunningTime="2026-07-15 20:37:39.80993911 +0000 UTC m=+2341.258312207"
Jul 15 20:37:48.657101 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:48.657059    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:48.657546 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:48.657140    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:48.670269 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:48.670240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:37:48.848062 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:37:48.848032    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:38:06.040413 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:06.040380    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"]
Jul 15 20:38:06.994583 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:06.994551    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"]
Jul 15 20:38:17.544018 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.543958    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"]
Jul 15 20:38:17.544480 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.544270    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv" podUID="16d826d9-b178-403a-93be-aadda86ed36f" containerName="maas-api" containerID="cri-o://f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1" gracePeriod=30
Jul 15 20:38:17.614643 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.614580    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv" podUID="16d826d9-b178-403a-93be-aadda86ed36f" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.45:8443/health\": dial tcp 10.133.0.45:8443: connect: connection refused"
Jul 15 20:38:17.798580 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.798508    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:38:17.825161 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.825124    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cqvh\" (UniqueName: \"kubernetes.io/projected/16d826d9-b178-403a-93be-aadda86ed36f-kube-api-access-8cqvh\") pod \"16d826d9-b178-403a-93be-aadda86ed36f\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") "
Jul 15 20:38:17.825347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.825201    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d826d9-b178-403a-93be-aadda86ed36f-openshift-service-ca\") pod \"16d826d9-b178-403a-93be-aadda86ed36f\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") "
Jul 15 20:38:17.825347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.825312    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16d826d9-b178-403a-93be-aadda86ed36f-maas-api-tls\") pod \"16d826d9-b178-403a-93be-aadda86ed36f\" (UID: \"16d826d9-b178-403a-93be-aadda86ed36f\") "
Jul 15 20:38:17.825597 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.825569    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16d826d9-b178-403a-93be-aadda86ed36f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16d826d9-b178-403a-93be-aadda86ed36f" (UID: "16d826d9-b178-403a-93be-aadda86ed36f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:38:17.827349 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.827321    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16d826d9-b178-403a-93be-aadda86ed36f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16d826d9-b178-403a-93be-aadda86ed36f" (UID: "16d826d9-b178-403a-93be-aadda86ed36f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:38:17.827473 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.827374    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d826d9-b178-403a-93be-aadda86ed36f-kube-api-access-8cqvh" (OuterVolumeSpecName: "kube-api-access-8cqvh") pod "16d826d9-b178-403a-93be-aadda86ed36f" (UID: "16d826d9-b178-403a-93be-aadda86ed36f"). InnerVolumeSpecName "kube-api-access-8cqvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:38:17.926185 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.926147    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cqvh\" (UniqueName: \"kubernetes.io/projected/16d826d9-b178-403a-93be-aadda86ed36f-kube-api-access-8cqvh\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:17.926185 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.926177    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d826d9-b178-403a-93be-aadda86ed36f-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:17.926185 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.926186    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16d826d9-b178-403a-93be-aadda86ed36f-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:17.980218 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.980184    2562 generic.go:356] "Generic (PLEG): container finished" podID="16d826d9-b178-403a-93be-aadda86ed36f" containerID="f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1" exitCode=0
Jul 15 20:38:17.980407 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.980234    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"
Jul 15 20:38:17.980407 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.980242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv" event={"ID":"16d826d9-b178-403a-93be-aadda86ed36f","Type":"ContainerDied","Data":"f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1"}
Jul 15 20:38:17.980407 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.980294    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv" event={"ID":"16d826d9-b178-403a-93be-aadda86ed36f","Type":"ContainerDied","Data":"a3f98a519be88e1dfb0a2a61c21785ec94c024c8e91e77d7c56fbf67cacee051"}
Jul 15 20:38:17.980407 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.980321    2562 scope.go:117] "RemoveContainer" containerID="f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1"
Jul 15 20:38:17.989541 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.989520    2562 scope.go:117] "RemoveContainer" containerID="f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1"
Jul 15 20:38:17.989857 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:38:17.989838    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1\": container with ID starting with f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1 not found: ID does not exist" containerID="f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1"
Jul 15 20:38:17.989906 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:17.989868    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1"} err="failed to get container status \"f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1\": rpc error: code = NotFound desc = could not find container \"f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1\": container with ID starting with f5bf6d39341404b83f6fce79a86bc625a07d44eefe20f4bc0ebd9520b54604a1 not found: ID does not exist"
Jul 15 20:38:18.002554 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:18.002521    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"]
Jul 15 20:38:18.007229 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:18.007206    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b06d27-79845489d7-fgwtv"]
Jul 15 20:38:19.078878 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:19.078843    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16d826d9-b178-403a-93be-aadda86ed36f" path="/var/lib/kubelet/pods/16d826d9-b178-403a-93be-aadda86ed36f/volumes"
Jul 15 20:38:21.061482 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.061440    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t" podUID="98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" containerName="main" containerID="cri-o://68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a" gracePeriod=45
Jul 15 20:38:21.313703 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.313637    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:38:21.355631 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.355605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-model-cache\") pod \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") "
Jul 15 20:38:21.355791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.355660    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tmp-dir\") pod \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") "
Jul 15 20:38:21.355791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.355704    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-home\") pod \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") "
Jul 15 20:38:21.355791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.355739    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-dshm\") pod \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") "
Jul 15 20:38:21.355791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.355785    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmltm\" (UniqueName: \"kubernetes.io/projected/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-kube-api-access-jmltm\") pod \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") "
Jul 15 20:38:21.356039 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.355812    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tls-certs\") pod \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\" (UID: \"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6\") "
Jul 15 20:38:21.356039 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.355871    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" (UID: "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:21.356039 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.356004    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" (UID: "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:21.356039 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.356018    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-home" (OuterVolumeSpecName: "home") pod "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" (UID: "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:21.356216 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.356164    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:21.356216 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.356180    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:21.356216 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.356196    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:21.358139 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.358107    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-dshm" (OuterVolumeSpecName: "dshm") pod "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" (UID: "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:21.358262 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.358147    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" (UID: "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:38:21.358262 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.358153    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-kube-api-access-jmltm" (OuterVolumeSpecName: "kube-api-access-jmltm") pod "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" (UID: "98b9897f-7b81-4bfc-b1d0-7c58618eb9d6"). InnerVolumeSpecName "kube-api-access-jmltm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:38:21.457305 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.457255    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:21.457305 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.457299    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmltm\" (UniqueName: \"kubernetes.io/projected/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-kube-api-access-jmltm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:21.457305 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:21.457310    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:22.001985 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.001943    2562 generic.go:356] "Generic (PLEG): container finished" podID="98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" containerID="68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a" exitCode=0
Jul 15 20:38:22.002151 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.002001    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"
Jul 15 20:38:22.002151 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.002008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t" event={"ID":"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6","Type":"ContainerDied","Data":"68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a"}
Jul 15 20:38:22.002151 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.002046    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t" event={"ID":"98b9897f-7b81-4bfc-b1d0-7c58618eb9d6","Type":"ContainerDied","Data":"f93de0e30e89db48dffe34ea65a92ed0c1dde5656f92d19105b76ae1391e33c1"}
Jul 15 20:38:22.002151 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.002064    2562 scope.go:117] "RemoveContainer" containerID="68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a"
Jul 15 20:38:22.012831 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.012800    2562 scope.go:117] "RemoveContainer" containerID="68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a"
Jul 15 20:38:22.013130 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:38:22.013106    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a\": container with ID starting with 68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a not found: ID does not exist" containerID="68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a"
Jul 15 20:38:22.013249 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.013135    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a"} err="failed to get container status \"68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a\": rpc error: code = NotFound desc = could not find container \"68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a\": container with ID starting with 68c7ed013e5c8ee9c1b64d66df0b6154efabf26df08ff9ed49e9b743a309ed2a not found: ID does not exist"
Jul 15 20:38:22.014627 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.014602    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2" podUID="1349e96f-c64c-4744-aac3-ef20f0b27389" containerName="main" containerID="cri-o://f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72" gracePeriod=45
Jul 15 20:38:22.027216 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.027195    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"]
Jul 15 20:38:22.033671 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.033649    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b06d27/test-model-b06d27-kserve-74bccc6487-7jz2t"]
Jul 15 20:38:22.275646 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.275620    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:38:22.367042 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367008    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-model-cache\") pod \"1349e96f-c64c-4744-aac3-ef20f0b27389\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") "
Jul 15 20:38:22.367210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367067    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1349e96f-c64c-4744-aac3-ef20f0b27389-tls-certs\") pod \"1349e96f-c64c-4744-aac3-ef20f0b27389\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") "
Jul 15 20:38:22.367210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367132    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-home\") pod \"1349e96f-c64c-4744-aac3-ef20f0b27389\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") "
Jul 15 20:38:22.367210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367158    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-dshm\") pod \"1349e96f-c64c-4744-aac3-ef20f0b27389\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") "
Jul 15 20:38:22.367210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367185    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlnnv\" (UniqueName: \"kubernetes.io/projected/1349e96f-c64c-4744-aac3-ef20f0b27389-kube-api-access-xlnnv\") pod \"1349e96f-c64c-4744-aac3-ef20f0b27389\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") "
Jul 15 20:38:22.367210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367209    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-tmp-dir\") pod \"1349e96f-c64c-4744-aac3-ef20f0b27389\" (UID: \"1349e96f-c64c-4744-aac3-ef20f0b27389\") "
Jul 15 20:38:22.367432 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367319    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-model-cache" (OuterVolumeSpecName: "model-cache") pod "1349e96f-c64c-4744-aac3-ef20f0b27389" (UID: "1349e96f-c64c-4744-aac3-ef20f0b27389"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:22.367497 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367456    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-home" (OuterVolumeSpecName: "home") pod "1349e96f-c64c-4744-aac3-ef20f0b27389" (UID: "1349e96f-c64c-4744-aac3-ef20f0b27389"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:22.367549 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367514    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:22.367602 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.367564    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1349e96f-c64c-4744-aac3-ef20f0b27389" (UID: "1349e96f-c64c-4744-aac3-ef20f0b27389"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:22.369463 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.369442    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-dshm" (OuterVolumeSpecName: "dshm") pod "1349e96f-c64c-4744-aac3-ef20f0b27389" (UID: "1349e96f-c64c-4744-aac3-ef20f0b27389"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:38:22.369463 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.369448    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1349e96f-c64c-4744-aac3-ef20f0b27389-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1349e96f-c64c-4744-aac3-ef20f0b27389" (UID: "1349e96f-c64c-4744-aac3-ef20f0b27389"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:38:22.369598 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.369515    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1349e96f-c64c-4744-aac3-ef20f0b27389-kube-api-access-xlnnv" (OuterVolumeSpecName: "kube-api-access-xlnnv") pod "1349e96f-c64c-4744-aac3-ef20f0b27389" (UID: "1349e96f-c64c-4744-aac3-ef20f0b27389"). InnerVolumeSpecName "kube-api-access-xlnnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:38:22.468599 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.468566    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1349e96f-c64c-4744-aac3-ef20f0b27389-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:22.468599 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.468593    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:22.468599 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.468601    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:22.468599 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.468610    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlnnv\" (UniqueName: \"kubernetes.io/projected/1349e96f-c64c-4744-aac3-ef20f0b27389-kube-api-access-xlnnv\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:22.468857 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:22.468620    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1349e96f-c64c-4744-aac3-ef20f0b27389-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:23.008594 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.008558    2562 generic.go:356] "Generic (PLEG): container finished" podID="1349e96f-c64c-4744-aac3-ef20f0b27389" containerID="f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72" exitCode=0
Jul 15 20:38:23.008798 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.008618    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"
Jul 15 20:38:23.008798 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.008643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2" event={"ID":"1349e96f-c64c-4744-aac3-ef20f0b27389","Type":"ContainerDied","Data":"f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72"}
Jul 15 20:38:23.008798 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.008689    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2" event={"ID":"1349e96f-c64c-4744-aac3-ef20f0b27389","Type":"ContainerDied","Data":"4c2ed1bcc8ccc8201e507cc9e04c5c958d68811739062a54a66e05bd3162978d"}
Jul 15 20:38:23.008798 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.008707    2562 scope.go:117] "RemoveContainer" containerID="f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72"
Jul 15 20:38:23.017732 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.017713    2562 scope.go:117] "RemoveContainer" containerID="f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72"
Jul 15 20:38:23.017988 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:38:23.017949    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72\": container with ID starting with f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72 not found: ID does not exist" containerID="f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72"
Jul 15 20:38:23.018099 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.017997    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72"} err="failed to get container status \"f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72\": rpc error: code = NotFound desc = could not find container \"f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72\": container with ID starting with f6de99fb8938cdadcb89325f7ad0e1f260222a9b7653e958d23b4ca140bafe72 not found: ID does not exist"
Jul 15 20:38:23.029657 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.029630    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"]
Jul 15 20:38:23.033686 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.033666    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e45884/test-model-e45884-kserve-745544b685-59sc2"]
Jul 15 20:38:23.078867 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.078837    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1349e96f-c64c-4744-aac3-ef20f0b27389" path="/var/lib/kubelet/pods/1349e96f-c64c-4744-aac3-ef20f0b27389/volumes"
Jul 15 20:38:23.079317 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:23.079299    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" path="/var/lib/kubelet/pods/98b9897f-7b81-4bfc-b1d0-7c58618eb9d6/volumes"
Jul 15 20:38:43.795326 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:43.795292    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"]
Jul 15 20:38:43.795841 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:43.795514    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl" podUID="9ed31653-94fc-4b4d-8480-72b08b1b788d" containerName="maas-api" containerID="cri-o://c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218" gracePeriod=30
Jul 15 20:38:44.245041 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.245014    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:38:44.370418 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.370328    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdm2v\" (UniqueName: \"kubernetes.io/projected/9ed31653-94fc-4b4d-8480-72b08b1b788d-kube-api-access-fdm2v\") pod \"9ed31653-94fc-4b4d-8480-72b08b1b788d\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") "
Jul 15 20:38:44.370586 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.370453    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed31653-94fc-4b4d-8480-72b08b1b788d-openshift-service-ca\") pod \"9ed31653-94fc-4b4d-8480-72b08b1b788d\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") "
Jul 15 20:38:44.370586 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.370503    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed31653-94fc-4b4d-8480-72b08b1b788d-maas-api-tls\") pod \"9ed31653-94fc-4b4d-8480-72b08b1b788d\" (UID: \"9ed31653-94fc-4b4d-8480-72b08b1b788d\") "
Jul 15 20:38:44.370904 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.370874    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed31653-94fc-4b4d-8480-72b08b1b788d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ed31653-94fc-4b4d-8480-72b08b1b788d" (UID: "9ed31653-94fc-4b4d-8480-72b08b1b788d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:38:44.372401 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.372378    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed31653-94fc-4b4d-8480-72b08b1b788d-kube-api-access-fdm2v" (OuterVolumeSpecName: "kube-api-access-fdm2v") pod "9ed31653-94fc-4b4d-8480-72b08b1b788d" (UID: "9ed31653-94fc-4b4d-8480-72b08b1b788d"). InnerVolumeSpecName "kube-api-access-fdm2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:38:44.372499 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.372477    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ed31653-94fc-4b4d-8480-72b08b1b788d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ed31653-94fc-4b4d-8480-72b08b1b788d" (UID: "9ed31653-94fc-4b4d-8480-72b08b1b788d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:38:44.471626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.471597    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ed31653-94fc-4b4d-8480-72b08b1b788d-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:44.471626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.471622    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ed31653-94fc-4b4d-8480-72b08b1b788d-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:44.471626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:44.471631    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdm2v\" (UniqueName: \"kubernetes.io/projected/9ed31653-94fc-4b4d-8480-72b08b1b788d-kube-api-access-fdm2v\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:38:45.112862 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.112831    2562 generic.go:356] "Generic (PLEG): container finished" podID="9ed31653-94fc-4b4d-8480-72b08b1b788d" containerID="c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218" exitCode=0
Jul 15 20:38:45.113253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.112872    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"
Jul 15 20:38:45.113253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.112911    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl" event={"ID":"9ed31653-94fc-4b4d-8480-72b08b1b788d","Type":"ContainerDied","Data":"c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218"}
Jul 15 20:38:45.113253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.112960    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl" event={"ID":"9ed31653-94fc-4b4d-8480-72b08b1b788d","Type":"ContainerDied","Data":"e94c2e7ecc27711928d146ba9a6b8dd8b27e34d2e3fd166671ca0f90be2754a4"}
Jul 15 20:38:45.113253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.112997    2562 scope.go:117] "RemoveContainer" containerID="c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218"
Jul 15 20:38:45.122207 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.122190    2562 scope.go:117] "RemoveContainer" containerID="c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218"
Jul 15 20:38:45.122799 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:38:45.122776    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218\": container with ID starting with c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218 not found: ID does not exist" containerID="c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218"
Jul 15 20:38:45.122933 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.122913    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218"} err="failed to get container status \"c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218\": rpc error: code = NotFound desc = could not find container \"c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218\": container with ID starting with c30162d8b1411055f06ae31d15be34e988ed65c876220692970462f6d7f34218 not found: ID does not exist"
Jul 15 20:38:45.132028 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.131994    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"]
Jul 15 20:38:45.136812 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:45.136791    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e45884-5448d69559-vkfxl"]
Jul 15 20:38:47.078417 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:38:47.078384    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed31653-94fc-4b4d-8480-72b08b1b788d" path="/var/lib/kubelet/pods/9ed31653-94fc-4b4d-8480-72b08b1b788d/volumes"
Jul 15 20:39:03.619945 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.619851    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"]
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620268    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d826d9-b178-403a-93be-aadda86ed36f" containerName="maas-api"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620280    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d826d9-b178-403a-93be-aadda86ed36f" containerName="maas-api"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620290    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" containerName="main"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620298    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" containerName="main"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620310    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed31653-94fc-4b4d-8480-72b08b1b788d" containerName="maas-api"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620316    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed31653-94fc-4b4d-8480-72b08b1b788d" containerName="maas-api"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620324    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1349e96f-c64c-4744-aac3-ef20f0b27389" containerName="main"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620330    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1349e96f-c64c-4744-aac3-ef20f0b27389" containerName="main"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620389    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b9897f-7b81-4bfc-b1d0-7c58618eb9d6" containerName="main"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620398    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d826d9-b178-403a-93be-aadda86ed36f" containerName="maas-api"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620404    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1349e96f-c64c-4744-aac3-ef20f0b27389" containerName="main"
Jul 15 20:39:03.620415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.620413    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed31653-94fc-4b4d-8480-72b08b1b788d" containerName="maas-api"
Jul 15 20:39:03.624564 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.624544    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.627070 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.627037    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4fa9c7-serving-cert\""
Jul 15 20:39:03.635793 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.635771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"]
Jul 15 20:39:03.647538 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.647504    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsgzn\" (UniqueName: \"kubernetes.io/projected/e767684b-a025-4d2b-95af-40d6970b1c56-kube-api-access-bsgzn\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.647691 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.647549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.647691 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.647629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e767684b-a025-4d2b-95af-40d6970b1c56-openshift-service-ca\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.748813 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.748771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsgzn\" (UniqueName: \"kubernetes.io/projected/e767684b-a025-4d2b-95af-40d6970b1c56-kube-api-access-bsgzn\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.748813 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.748818    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.749123 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.748880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e767684b-a025-4d2b-95af-40d6970b1c56-openshift-service-ca\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.749123 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:39:03.749003    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-serving-cert: secret "maas-api-e2e-api-a-4fa9c7-serving-cert" not found
Jul 15 20:39:03.749123 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:39:03.749064    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls podName:e767684b-a025-4d2b-95af-40d6970b1c56 nodeName:}" failed. No retries permitted until 2026-07-15 20:39:04.249048342 +0000 UTC m=+2425.697421415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls") pod "maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b" (UID: "e767684b-a025-4d2b-95af-40d6970b1c56") : secret "maas-api-e2e-api-a-4fa9c7-serving-cert" not found
Jul 15 20:39:03.749665 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.749647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e767684b-a025-4d2b-95af-40d6970b1c56-openshift-service-ca\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:03.758173 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:03.758153    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsgzn\" (UniqueName: \"kubernetes.io/projected/e767684b-a025-4d2b-95af-40d6970b1c56-kube-api-access-bsgzn\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:04.254144 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:04.254111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:04.256654 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:04.256631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls\") pod \"maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:04.535746 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:04.535646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:04.673277 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:04.673214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"]
Jul 15 20:39:04.675537 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:39:04.675510    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode767684b_a025_4d2b_95af_40d6970b1c56.slice/crio-6663385e0b23b4a1a1e4906d5cc00b9a06d1a69a0838cf9f6b20f29e080ea8ae WatchSource:0}: Error finding container 6663385e0b23b4a1a1e4906d5cc00b9a06d1a69a0838cf9f6b20f29e080ea8ae: Status 404 returned error can't find the container with id 6663385e0b23b4a1a1e4906d5cc00b9a06d1a69a0838cf9f6b20f29e080ea8ae
Jul 15 20:39:04.677065 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:04.677042    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:39:05.204201 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:05.204163    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b" event={"ID":"e767684b-a025-4d2b-95af-40d6970b1c56","Type":"ContainerStarted","Data":"6663385e0b23b4a1a1e4906d5cc00b9a06d1a69a0838cf9f6b20f29e080ea8ae"}
Jul 15 20:39:06.210472 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:06.210440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b" event={"ID":"e767684b-a025-4d2b-95af-40d6970b1c56","Type":"ContainerStarted","Data":"e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8"}
Jul 15 20:39:06.211268 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:06.210573    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:06.231770 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:06.231723    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b" podStartSLOduration=2.696123853 podStartE2EDuration="3.231704881s" totalImagesPullingTime="535.581028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:39:06.229672477 +0000 UTC m=+2427.678045574" watchObservedRunningTime="2026-07-15 20:39:06.231704881 +0000 UTC m=+2427.680078399"
Jul 15 20:39:12.221582 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:12.221552    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:34.875551 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:34.875515    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"]
Jul 15 20:39:34.875994 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:34.875720    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b" podUID="e767684b-a025-4d2b-95af-40d6970b1c56" containerName="maas-api" containerID="cri-o://e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8" gracePeriod=30
Jul 15 20:39:35.119160 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.119130    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:35.150347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.150264    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls\") pod \"e767684b-a025-4d2b-95af-40d6970b1c56\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") "
Jul 15 20:39:35.150347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.150314    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e767684b-a025-4d2b-95af-40d6970b1c56-openshift-service-ca\") pod \"e767684b-a025-4d2b-95af-40d6970b1c56\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") "
Jul 15 20:39:35.150575 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.150369    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsgzn\" (UniqueName: \"kubernetes.io/projected/e767684b-a025-4d2b-95af-40d6970b1c56-kube-api-access-bsgzn\") pod \"e767684b-a025-4d2b-95af-40d6970b1c56\" (UID: \"e767684b-a025-4d2b-95af-40d6970b1c56\") "
Jul 15 20:39:35.150728 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.150700    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e767684b-a025-4d2b-95af-40d6970b1c56-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e767684b-a025-4d2b-95af-40d6970b1c56" (UID: "e767684b-a025-4d2b-95af-40d6970b1c56"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:39:35.152461 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.152434    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e767684b-a025-4d2b-95af-40d6970b1c56" (UID: "e767684b-a025-4d2b-95af-40d6970b1c56"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:39:35.152585 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.152507    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e767684b-a025-4d2b-95af-40d6970b1c56-kube-api-access-bsgzn" (OuterVolumeSpecName: "kube-api-access-bsgzn") pod "e767684b-a025-4d2b-95af-40d6970b1c56" (UID: "e767684b-a025-4d2b-95af-40d6970b1c56"). InnerVolumeSpecName "kube-api-access-bsgzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:39:35.251638 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.251610    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e767684b-a025-4d2b-95af-40d6970b1c56-openshift-service-ca\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:39:35.251638 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.251634    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsgzn\" (UniqueName: \"kubernetes.io/projected/e767684b-a025-4d2b-95af-40d6970b1c56-kube-api-access-bsgzn\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:39:35.251785 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.251643    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e767684b-a025-4d2b-95af-40d6970b1c56-maas-api-tls\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:39:35.347057 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.347028    2562 generic.go:356] "Generic (PLEG): container finished" podID="e767684b-a025-4d2b-95af-40d6970b1c56" containerID="e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8" exitCode=0
Jul 15 20:39:35.347190 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.347073    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"
Jul 15 20:39:35.347190 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.347110    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b" event={"ID":"e767684b-a025-4d2b-95af-40d6970b1c56","Type":"ContainerDied","Data":"e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8"}
Jul 15 20:39:35.347190 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.347164    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b" event={"ID":"e767684b-a025-4d2b-95af-40d6970b1c56","Type":"ContainerDied","Data":"6663385e0b23b4a1a1e4906d5cc00b9a06d1a69a0838cf9f6b20f29e080ea8ae"}
Jul 15 20:39:35.347327 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.347193    2562 scope.go:117] "RemoveContainer" containerID="e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8"
Jul 15 20:39:35.356622 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.356605    2562 scope.go:117] "RemoveContainer" containerID="e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8"
Jul 15 20:39:35.356886 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:39:35.356865    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8\": container with ID starting with e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8 not found: ID does not exist" containerID="e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8"
Jul 15 20:39:35.356959 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.356895    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8"} err="failed to get container status \"e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8\": rpc error: code = NotFound desc = could not find container \"e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8\": container with ID starting with e254b7b12737437a8f35b12b7da6531b9d3dff31416f233daba9c3c84059d8b8 not found: ID does not exist"
Jul 15 20:39:35.371377 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.371351    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"]
Jul 15 20:39:35.374517 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:35.374497    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4fa9c7-545c7948dc-8f78b"]
Jul 15 20:39:37.079081 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:39:37.079051    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e767684b-a025-4d2b-95af-40d6970b1c56" path="/var/lib/kubelet/pods/e767684b-a025-4d2b-95af-40d6970b1c56/volumes"
Jul 15 20:41:12.446575 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.446543    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"]
Jul 15 20:41:12.447198 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.447165    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e767684b-a025-4d2b-95af-40d6970b1c56" containerName="maas-api"
Jul 15 20:41:12.447198 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.447185    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e767684b-a025-4d2b-95af-40d6970b1c56" containerName="maas-api"
Jul 15 20:41:12.447319 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.447310    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e767684b-a025-4d2b-95af-40d6970b1c56" containerName="maas-api"
Jul 15 20:41:12.450371 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.450351    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.454061 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.454034    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"openshift-service-ca.crt\""
Jul 15 20:41:12.454061 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.454034    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"default-dockercfg-8s9fq\""
Jul 15 20:41:12.454256 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.454034    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"auth-test-model-724347-kserve-self-signed-certs\""
Jul 15 20:41:12.454256 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.454173    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"kube-root-ca.crt\""
Jul 15 20:41:12.460543 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.460523    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"]
Jul 15 20:41:12.522257 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.522226    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-dshm\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.522415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.522264    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-tmp-dir\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.522415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.522346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr6ct\" (UniqueName: \"kubernetes.io/projected/12530845-f340-4cf0-810f-a862fcd7bfc8-kube-api-access-nr6ct\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.522415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.522398    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-model-cache\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.522526 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.522423    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-home\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.522526 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.522475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12530845-f340-4cf0-810f-a862fcd7bfc8-tls-certs\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623239 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12530845-f340-4cf0-810f-a862fcd7bfc8-tls-certs\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623410 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-dshm\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623410 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-tmp-dir\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623410 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr6ct\" (UniqueName: \"kubernetes.io/projected/12530845-f340-4cf0-810f-a862fcd7bfc8-kube-api-access-nr6ct\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623410 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623339    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-model-cache\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623410 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623357    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-home\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623747 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-model-cache\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623811 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-tmp-dir\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.623811 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.623778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-home\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.625469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.625448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-dshm\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.625763 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.625746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12530845-f340-4cf0-810f-a862fcd7bfc8-tls-certs\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.631621 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.631604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr6ct\" (UniqueName: \"kubernetes.io/projected/12530845-f340-4cf0-810f-a862fcd7bfc8-kube-api-access-nr6ct\") pod \"auth-test-model-724347-kserve-5678b7bd75-vjczj\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") " pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.777220 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.777126    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:12.909231 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:12.909199    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"]
Jul 15 20:41:12.912413 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:41:12.912385    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12530845_f340_4cf0_810f_a862fcd7bfc8.slice/crio-97f899321806e68908f90cd36402692e76891a9e9a74e79e430c6cdd4f4c3c4e WatchSource:0}: Error finding container 97f899321806e68908f90cd36402692e76891a9e9a74e79e430c6cdd4f4c3c4e: Status 404 returned error can't find the container with id 97f899321806e68908f90cd36402692e76891a9e9a74e79e430c6cdd4f4c3c4e
Jul 15 20:41:13.799037 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:13.798999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj" event={"ID":"12530845-f340-4cf0-810f-a862fcd7bfc8","Type":"ContainerStarted","Data":"0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62"}
Jul 15 20:41:13.799481 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:13.799045    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj" event={"ID":"12530845-f340-4cf0-810f-a862fcd7bfc8","Type":"ContainerStarted","Data":"97f899321806e68908f90cd36402692e76891a9e9a74e79e430c6cdd4f4c3c4e"}
Jul 15 20:41:13.817700 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:13.817645    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj" podStartSLOduration=1.817630301 podStartE2EDuration="1.817630301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:41:13.816092388 +0000 UTC m=+2555.264465484" watchObservedRunningTime="2026-07-15 20:41:13.817630301 +0000 UTC m=+2555.266003396"
Jul 15 20:41:22.777861 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:22.777828    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:22.778376 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:22.777916    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:22.790337 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:22.790309    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:22.851846 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:22.851821    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:41:27.152460 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.152431    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"]
Jul 15 20:41:27.155743 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.155727    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.158666 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.158642    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"openshift-service-ca.crt\""
Jul 15 20:41:27.160090 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.160074    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"kube-root-ca.crt\""
Jul 15 20:41:27.160187 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.160113    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"default-dockercfg-n8667\""
Jul 15 20:41:27.160187 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.160116    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"auth-test-model-5292f5-kserve-self-signed-certs\""
Jul 15 20:41:27.166179 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.166158    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"]
Jul 15 20:41:27.261197 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.261160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-home\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.261374 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.261231    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-dshm\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.261374 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.261278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-model-cache\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.261498 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.261374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130064cd-392f-4a78-97f8-d7f7ccab0385-tls-certs\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.261498 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.261411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-tmp-dir\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.261498 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.261467    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-268hc\" (UniqueName: \"kubernetes.io/projected/130064cd-392f-4a78-97f8-d7f7ccab0385-kube-api-access-268hc\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.361912 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.361878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-268hc\" (UniqueName: \"kubernetes.io/projected/130064cd-392f-4a78-97f8-d7f7ccab0385-kube-api-access-268hc\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362095 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.361989    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-home\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362095 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.362040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-dshm\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362095 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.362072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-model-cache\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362257 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.362121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130064cd-392f-4a78-97f8-d7f7ccab0385-tls-certs\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362257 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.362149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-tmp-dir\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362424 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.362396    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-home\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.362458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-model-cache\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.362571 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.362548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-tmp-dir\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.364217 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.364194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-dshm\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.364524 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.364500    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130064cd-392f-4a78-97f8-d7f7ccab0385-tls-certs\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.371273 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.371253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-268hc\" (UniqueName: \"kubernetes.io/projected/130064cd-392f-4a78-97f8-d7f7ccab0385-kube-api-access-268hc\") pod \"auth-test-model-5292f5-kserve-649f669bbc-tnvlc\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") " pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.466210 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.466184    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:27.598483 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.598459    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"]
Jul 15 20:41:27.600920 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:41:27.600893    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod130064cd_392f_4a78_97f8_d7f7ccab0385.slice/crio-51d62004d9ec11467297a3a961c72c95370e868cc7c3a4f9298217db15bf3e22 WatchSource:0}: Error finding container 51d62004d9ec11467297a3a961c72c95370e868cc7c3a4f9298217db15bf3e22: Status 404 returned error can't find the container with id 51d62004d9ec11467297a3a961c72c95370e868cc7c3a4f9298217db15bf3e22
Jul 15 20:41:27.864910 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.864818    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc" event={"ID":"130064cd-392f-4a78-97f8-d7f7ccab0385","Type":"ContainerStarted","Data":"9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282"}
Jul 15 20:41:27.864910 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.864862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc" event={"ID":"130064cd-392f-4a78-97f8-d7f7ccab0385","Type":"ContainerStarted","Data":"51d62004d9ec11467297a3a961c72c95370e868cc7c3a4f9298217db15bf3e22"}
Jul 15 20:41:27.883860 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:27.883815    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc" podStartSLOduration=0.883800448 podStartE2EDuration="883.800448ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:41:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:41:27.883100764 +0000 UTC m=+2569.331473874" watchObservedRunningTime="2026-07-15 20:41:27.883800448 +0000 UTC m=+2569.332173543"
Jul 15 20:41:37.466941 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:37.466906    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:37.467356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:37.467049    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:37.479426 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:37.479399    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:41:37.921461 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:41:37.921382    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:43:35.627754 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:35.627669    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"]
Jul 15 20:43:36.730791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:36.730753    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"]
Jul 15 20:43:46.855105 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.855071    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"]
Jul 15 20:43:46.858490 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.858473    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:46.861186 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.861166    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"sub-test-model-724347-kserve-self-signed-certs\""
Jul 15 20:43:46.867920 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.867899    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"]
Jul 15 20:43:46.960183 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.960153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-dshm\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:46.960183 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.960187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-home\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:46.960392 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.960205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52bf2207-e86a-4f23-b423-10e83de2b4c6-tls-certs\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:46.960392 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.960287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-model-cache\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:46.960392 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.960302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-tmp-dir\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:46.960392 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:46.960336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tns2c\" (UniqueName: \"kubernetes.io/projected/52bf2207-e86a-4f23-b423-10e83de2b4c6-kube-api-access-tns2c\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.061684 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.061652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-model-cache\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.061684 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.061691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-tmp-dir\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.061887 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.061813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tns2c\" (UniqueName: \"kubernetes.io/projected/52bf2207-e86a-4f23-b423-10e83de2b4c6-kube-api-access-tns2c\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.061961 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.061949    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-dshm\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.062033 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.061998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-home\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.062033 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.062015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52bf2207-e86a-4f23-b423-10e83de2b4c6-tls-certs\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.062203 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.062101    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-tmp-dir\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.062203 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.062117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-model-cache\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.062357 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.062334    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-home\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.064282 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.064251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-dshm\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.064393 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.064374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52bf2207-e86a-4f23-b423-10e83de2b4c6-tls-certs\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.070744 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.070724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tns2c\" (UniqueName: \"kubernetes.io/projected/52bf2207-e86a-4f23-b423-10e83de2b4c6-kube-api-access-tns2c\") pod \"sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") " pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.167823 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.167743    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:47.303008 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.302960    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"]
Jul 15 20:43:47.306346 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:43:47.306314    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52bf2207_e86a_4f23_b423_10e83de2b4c6.slice/crio-03435906be4c7ee8900f4f78e9a0ec0fde7d575e35948af9021781c3887ddda3 WatchSource:0}: Error finding container 03435906be4c7ee8900f4f78e9a0ec0fde7d575e35948af9021781c3887ddda3: Status 404 returned error can't find the container with id 03435906be4c7ee8900f4f78e9a0ec0fde7d575e35948af9021781c3887ddda3
Jul 15 20:43:47.520529 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.520489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c" event={"ID":"52bf2207-e86a-4f23-b423-10e83de2b4c6","Type":"ContainerStarted","Data":"2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816"}
Jul 15 20:43:47.520529 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.520530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c" event={"ID":"52bf2207-e86a-4f23-b423-10e83de2b4c6","Type":"ContainerStarted","Data":"03435906be4c7ee8900f4f78e9a0ec0fde7d575e35948af9021781c3887ddda3"}
Jul 15 20:43:47.540400 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:47.540346    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c" podStartSLOduration=1.540330563 podStartE2EDuration="1.540330563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:43:47.538761351 +0000 UTC m=+2708.987134447" watchObservedRunningTime="2026-07-15 20:43:47.540330563 +0000 UTC m=+2708.988703715"
Jul 15 20:43:50.648144 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.648078    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj" podUID="12530845-f340-4cf0-810f-a862fcd7bfc8" containerName="main" containerID="cri-o://0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62" gracePeriod=45
Jul 15 20:43:50.895863 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.895834    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:43:50.994444 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994413    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12530845-f340-4cf0-810f-a862fcd7bfc8-tls-certs\") pod \"12530845-f340-4cf0-810f-a862fcd7bfc8\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") "
Jul 15 20:43:50.994634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994460    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-dshm\") pod \"12530845-f340-4cf0-810f-a862fcd7bfc8\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") "
Jul 15 20:43:50.994634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994493    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-model-cache\") pod \"12530845-f340-4cf0-810f-a862fcd7bfc8\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") "
Jul 15 20:43:50.994634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994515    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr6ct\" (UniqueName: \"kubernetes.io/projected/12530845-f340-4cf0-810f-a862fcd7bfc8-kube-api-access-nr6ct\") pod \"12530845-f340-4cf0-810f-a862fcd7bfc8\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") "
Jul 15 20:43:50.994634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994567    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-tmp-dir\") pod \"12530845-f340-4cf0-810f-a862fcd7bfc8\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") "
Jul 15 20:43:50.994634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994584    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-home\") pod \"12530845-f340-4cf0-810f-a862fcd7bfc8\" (UID: \"12530845-f340-4cf0-810f-a862fcd7bfc8\") "
Jul 15 20:43:50.994945 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994843    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-model-cache" (OuterVolumeSpecName: "model-cache") pod "12530845-f340-4cf0-810f-a862fcd7bfc8" (UID: "12530845-f340-4cf0-810f-a862fcd7bfc8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:50.994945 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994854    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "12530845-f340-4cf0-810f-a862fcd7bfc8" (UID: "12530845-f340-4cf0-810f-a862fcd7bfc8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:50.995087 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.994940    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-home" (OuterVolumeSpecName: "home") pod "12530845-f340-4cf0-810f-a862fcd7bfc8" (UID: "12530845-f340-4cf0-810f-a862fcd7bfc8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:50.996787 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.996763    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12530845-f340-4cf0-810f-a862fcd7bfc8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "12530845-f340-4cf0-810f-a862fcd7bfc8" (UID: "12530845-f340-4cf0-810f-a862fcd7bfc8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:43:50.996787 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.996761    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-dshm" (OuterVolumeSpecName: "dshm") pod "12530845-f340-4cf0-810f-a862fcd7bfc8" (UID: "12530845-f340-4cf0-810f-a862fcd7bfc8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:50.996936 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:50.996824    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12530845-f340-4cf0-810f-a862fcd7bfc8-kube-api-access-nr6ct" (OuterVolumeSpecName: "kube-api-access-nr6ct") pod "12530845-f340-4cf0-810f-a862fcd7bfc8" (UID: "12530845-f340-4cf0-810f-a862fcd7bfc8"). InnerVolumeSpecName "kube-api-access-nr6ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:43:51.095346 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.095321    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:51.095346 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.095344    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:51.095495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.095353    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12530845-f340-4cf0-810f-a862fcd7bfc8-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:51.095495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.095362    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:51.095495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.095370    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12530845-f340-4cf0-810f-a862fcd7bfc8-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:51.095495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.095377    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr6ct\" (UniqueName: \"kubernetes.io/projected/12530845-f340-4cf0-810f-a862fcd7bfc8-kube-api-access-nr6ct\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:51.539086 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.539051    2562 generic.go:356] "Generic (PLEG): container finished" podID="12530845-f340-4cf0-810f-a862fcd7bfc8" containerID="0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62" exitCode=0
Jul 15 20:43:51.539268 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.539096    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"
Jul 15 20:43:51.539268 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.539149    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj" event={"ID":"12530845-f340-4cf0-810f-a862fcd7bfc8","Type":"ContainerDied","Data":"0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62"}
Jul 15 20:43:51.539268 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.539202    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj" event={"ID":"12530845-f340-4cf0-810f-a862fcd7bfc8","Type":"ContainerDied","Data":"97f899321806e68908f90cd36402692e76891a9e9a74e79e430c6cdd4f4c3c4e"}
Jul 15 20:43:51.539268 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.539224    2562 scope.go:117] "RemoveContainer" containerID="0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62"
Jul 15 20:43:51.547486 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.547476    2562 scope.go:117] "RemoveContainer" containerID="0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62"
Jul 15 20:43:51.547762 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:43:51.547745    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62\": container with ID starting with 0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62 not found: ID does not exist" containerID="0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62"
Jul 15 20:43:51.547809 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.547771    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62"} err="failed to get container status \"0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62\": rpc error: code = NotFound desc = could not find container \"0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62\": container with ID starting with 0f25231d71baa35ebf8e28851d7d87a29fe673382a165f492480e5e2e9025d62 not found: ID does not exist"
Jul 15 20:43:51.555828 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.555804    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"]
Jul 15 20:43:51.559441 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.559420    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-724347/auth-test-model-724347-kserve-5678b7bd75-vjczj"]
Jul 15 20:43:51.751100 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:51.751051    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc" podUID="130064cd-392f-4a78-97f8-d7f7ccab0385" containerName="main" containerID="cri-o://9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282" gracePeriod=45
Jul 15 20:43:52.001936 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.001913    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:43:52.106086 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106056    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-tmp-dir\") pod \"130064cd-392f-4a78-97f8-d7f7ccab0385\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") "
Jul 15 20:43:52.106236 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106121    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-home\") pod \"130064cd-392f-4a78-97f8-d7f7ccab0385\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") "
Jul 15 20:43:52.106236 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106166    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130064cd-392f-4a78-97f8-d7f7ccab0385-tls-certs\") pod \"130064cd-392f-4a78-97f8-d7f7ccab0385\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") "
Jul 15 20:43:52.106236 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106193    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-268hc\" (UniqueName: \"kubernetes.io/projected/130064cd-392f-4a78-97f8-d7f7ccab0385-kube-api-access-268hc\") pod \"130064cd-392f-4a78-97f8-d7f7ccab0385\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") "
Jul 15 20:43:52.106236 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106229    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-model-cache\") pod \"130064cd-392f-4a78-97f8-d7f7ccab0385\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") "
Jul 15 20:43:52.106454 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106292    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-dshm\") pod \"130064cd-392f-4a78-97f8-d7f7ccab0385\" (UID: \"130064cd-392f-4a78-97f8-d7f7ccab0385\") "
Jul 15 20:43:52.106454 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106317    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "130064cd-392f-4a78-97f8-d7f7ccab0385" (UID: "130064cd-392f-4a78-97f8-d7f7ccab0385"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:52.106454 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106355    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-home" (OuterVolumeSpecName: "home") pod "130064cd-392f-4a78-97f8-d7f7ccab0385" (UID: "130064cd-392f-4a78-97f8-d7f7ccab0385"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:52.106593 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106470    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-model-cache" (OuterVolumeSpecName: "model-cache") pod "130064cd-392f-4a78-97f8-d7f7ccab0385" (UID: "130064cd-392f-4a78-97f8-d7f7ccab0385"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:52.106631 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106597    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:52.106631 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106608    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:52.106631 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.106615    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:52.108422 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.108395    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-dshm" (OuterVolumeSpecName: "dshm") pod "130064cd-392f-4a78-97f8-d7f7ccab0385" (UID: "130064cd-392f-4a78-97f8-d7f7ccab0385"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:43:52.108422 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.108416    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/130064cd-392f-4a78-97f8-d7f7ccab0385-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "130064cd-392f-4a78-97f8-d7f7ccab0385" (UID: "130064cd-392f-4a78-97f8-d7f7ccab0385"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:43:52.108565 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.108491    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/130064cd-392f-4a78-97f8-d7f7ccab0385-kube-api-access-268hc" (OuterVolumeSpecName: "kube-api-access-268hc") pod "130064cd-392f-4a78-97f8-d7f7ccab0385" (UID: "130064cd-392f-4a78-97f8-d7f7ccab0385"). InnerVolumeSpecName "kube-api-access-268hc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:43:52.207450 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.207419    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130064cd-392f-4a78-97f8-d7f7ccab0385-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:52.207450 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.207446    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-268hc\" (UniqueName: \"kubernetes.io/projected/130064cd-392f-4a78-97f8-d7f7ccab0385-kube-api-access-268hc\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:52.207619 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.207459    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130064cd-392f-4a78-97f8-d7f7ccab0385-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:43:52.546247 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.546151    2562 generic.go:356] "Generic (PLEG): container finished" podID="130064cd-392f-4a78-97f8-d7f7ccab0385" containerID="9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282" exitCode=0
Jul 15 20:43:52.546247 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.546184    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"
Jul 15 20:43:52.546486 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.546242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc" event={"ID":"130064cd-392f-4a78-97f8-d7f7ccab0385","Type":"ContainerDied","Data":"9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282"}
Jul 15 20:43:52.546486 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.546281    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc" event={"ID":"130064cd-392f-4a78-97f8-d7f7ccab0385","Type":"ContainerDied","Data":"51d62004d9ec11467297a3a961c72c95370e868cc7c3a4f9298217db15bf3e22"}
Jul 15 20:43:52.546486 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.546297    2562 scope.go:117] "RemoveContainer" containerID="9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282"
Jul 15 20:43:52.555328 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.555311    2562 scope.go:117] "RemoveContainer" containerID="9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282"
Jul 15 20:43:52.555566 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:43:52.555548    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282\": container with ID starting with 9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282 not found: ID does not exist" containerID="9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282"
Jul 15 20:43:52.555632 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.555573    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282"} err="failed to get container status \"9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282\": rpc error: code = NotFound desc = could not find container \"9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282\": container with ID starting with 9533d7e8c0d58f2fb4a73a58007496ea25ffba88451159910721b43e90bee282 not found: ID does not exist"
Jul 15 20:43:52.567425 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.567404    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"]
Jul 15 20:43:52.571490 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:52.571472    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/auth-test-model-5292f5-kserve-649f669bbc-tnvlc"]
Jul 15 20:43:53.078864 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:53.078835    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12530845-f340-4cf0-810f-a862fcd7bfc8" path="/var/lib/kubelet/pods/12530845-f340-4cf0-810f-a862fcd7bfc8/volumes"
Jul 15 20:43:53.079260 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:53.079246    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="130064cd-392f-4a78-97f8-d7f7ccab0385" path="/var/lib/kubelet/pods/130064cd-392f-4a78-97f8-d7f7ccab0385/volumes"
Jul 15 20:43:57.168817 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:57.168782    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:57.169302 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:57.168829    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:57.181134 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:57.181106    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:43:57.580032 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:43:57.579990    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:44:14.640182 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.640148    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"]
Jul 15 20:44:14.640751 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.640694    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="130064cd-392f-4a78-97f8-d7f7ccab0385" containerName="main"
Jul 15 20:44:14.640751 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.640715    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="130064cd-392f-4a78-97f8-d7f7ccab0385" containerName="main"
Jul 15 20:44:14.640751 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.640729    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12530845-f340-4cf0-810f-a862fcd7bfc8" containerName="main"
Jul 15 20:44:14.640751 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.640737    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12530845-f340-4cf0-810f-a862fcd7bfc8" containerName="main"
Jul 15 20:44:14.640988 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.640845    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="130064cd-392f-4a78-97f8-d7f7ccab0385" containerName="main"
Jul 15 20:44:14.640988 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.640856    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="12530845-f340-4cf0-810f-a862fcd7bfc8" containerName="main"
Jul 15 20:44:14.645662 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.645642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.658772 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.649897    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"kube-root-ca.crt\""
Jul 15 20:44:14.658772 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.650074    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"sub-test-model-5292f5-kserve-self-signed-certs\""
Jul 15 20:44:14.658772 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.650425    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"default-dockercfg-n8667\""
Jul 15 20:44:14.658772 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.650802    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"openshift-service-ca.crt\""
Jul 15 20:44:14.658772 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.657880    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"]
Jul 15 20:44:14.810913 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.810875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5db649-f0e1-4997-bab7-36791125a75e-tls-certs\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.810913 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.810918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-home\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.811148 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.810961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9spc\" (UniqueName: \"kubernetes.io/projected/6f5db649-f0e1-4997-bab7-36791125a75e-kube-api-access-g9spc\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.811148 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.811038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-model-cache\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.811148 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.811061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-dshm\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.811148 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.811093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-tmp-dir\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912427 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-home\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912427 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9spc\" (UniqueName: \"kubernetes.io/projected/6f5db649-f0e1-4997-bab7-36791125a75e-kube-api-access-g9spc\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912647 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-model-cache\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912647 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912471    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-dshm\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912647 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-tmp-dir\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912647 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5db649-f0e1-4997-bab7-36791125a75e-tls-certs\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912832 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-home\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912891 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-tmp-dir\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.912891 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.912874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-model-cache\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.914634 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.914610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-dshm\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.914995 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.914961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5db649-f0e1-4997-bab7-36791125a75e-tls-certs\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.921760 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.921741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9spc\" (UniqueName: \"kubernetes.io/projected/6f5db649-f0e1-4997-bab7-36791125a75e-kube-api-access-g9spc\") pod \"sub-test-model-5292f5-kserve-5c676d447c-jb8wm\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") " pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:14.965852 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:14.965824    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:15.098415 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:15.098381    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"]
Jul 15 20:44:15.100980 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:44:15.100943    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f5db649_f0e1_4997_bab7_36791125a75e.slice/crio-b20216bcda3a7ea3c1136f556a8f22f5f3d72a5415fc255fb8b8be84ac334891 WatchSource:0}: Error finding container b20216bcda3a7ea3c1136f556a8f22f5f3d72a5415fc255fb8b8be84ac334891: Status 404 returned error can't find the container with id b20216bcda3a7ea3c1136f556a8f22f5f3d72a5415fc255fb8b8be84ac334891
Jul 15 20:44:15.665796 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:15.665759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm" event={"ID":"6f5db649-f0e1-4997-bab7-36791125a75e","Type":"ContainerStarted","Data":"71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf"}
Jul 15 20:44:15.665796 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:15.665804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm" event={"ID":"6f5db649-f0e1-4997-bab7-36791125a75e","Type":"ContainerStarted","Data":"b20216bcda3a7ea3c1136f556a8f22f5f3d72a5415fc255fb8b8be84ac334891"}
Jul 15 20:44:15.685446 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:15.685396    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm" podStartSLOduration=1.68538011 podStartE2EDuration="1.68538011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:44:15.683724652 +0000 UTC m=+2737.132097748" watchObservedRunningTime="2026-07-15 20:44:15.68538011 +0000 UTC m=+2737.133753204"
Jul 15 20:44:24.966511 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:24.966475    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:24.966936 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:24.966527    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:24.978901 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:24.978880    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:44:25.729072 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:44:25.729036    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:45:01.194076 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:01.193869    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"]
Jul 15 20:45:02.235836 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:02.235806    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"]
Jul 15 20:45:16.215012 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.214933    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c" podUID="52bf2207-e86a-4f23-b423-10e83de2b4c6" containerName="main" containerID="cri-o://2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816" gracePeriod=45
Jul 15 20:45:16.477677 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.477613    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:45:16.574101 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574060    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-home\") pod \"52bf2207-e86a-4f23-b423-10e83de2b4c6\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") "
Jul 15 20:45:16.574276 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574117    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-dshm\") pod \"52bf2207-e86a-4f23-b423-10e83de2b4c6\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") "
Jul 15 20:45:16.574276 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574144    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tns2c\" (UniqueName: \"kubernetes.io/projected/52bf2207-e86a-4f23-b423-10e83de2b4c6-kube-api-access-tns2c\") pod \"52bf2207-e86a-4f23-b423-10e83de2b4c6\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") "
Jul 15 20:45:16.574276 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574165    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-model-cache\") pod \"52bf2207-e86a-4f23-b423-10e83de2b4c6\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") "
Jul 15 20:45:16.574407 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574341    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-home" (OuterVolumeSpecName: "home") pod "52bf2207-e86a-4f23-b423-10e83de2b4c6" (UID: "52bf2207-e86a-4f23-b423-10e83de2b4c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:16.574407 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574345    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52bf2207-e86a-4f23-b423-10e83de2b4c6-tls-certs\") pod \"52bf2207-e86a-4f23-b423-10e83de2b4c6\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") "
Jul 15 20:45:16.574483 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-tmp-dir\") pod \"52bf2207-e86a-4f23-b423-10e83de2b4c6\" (UID: \"52bf2207-e86a-4f23-b423-10e83de2b4c6\") "
Jul 15 20:45:16.574483 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574444    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "52bf2207-e86a-4f23-b423-10e83de2b4c6" (UID: "52bf2207-e86a-4f23-b423-10e83de2b4c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:16.574700 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574670    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52bf2207-e86a-4f23-b423-10e83de2b4c6" (UID: "52bf2207-e86a-4f23-b423-10e83de2b4c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:16.574811 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574795    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:16.574873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574818    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:16.574873 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.574831    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:16.576468 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.576447    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52bf2207-e86a-4f23-b423-10e83de2b4c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52bf2207-e86a-4f23-b423-10e83de2b4c6" (UID: "52bf2207-e86a-4f23-b423-10e83de2b4c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:45:16.576729 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.576710    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-dshm" (OuterVolumeSpecName: "dshm") pod "52bf2207-e86a-4f23-b423-10e83de2b4c6" (UID: "52bf2207-e86a-4f23-b423-10e83de2b4c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:16.576782 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.576730    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52bf2207-e86a-4f23-b423-10e83de2b4c6-kube-api-access-tns2c" (OuterVolumeSpecName: "kube-api-access-tns2c") pod "52bf2207-e86a-4f23-b423-10e83de2b4c6" (UID: "52bf2207-e86a-4f23-b423-10e83de2b4c6"). InnerVolumeSpecName "kube-api-access-tns2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:45:16.676132 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.676099    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52bf2207-e86a-4f23-b423-10e83de2b4c6-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:16.676132 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.676126    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52bf2207-e86a-4f23-b423-10e83de2b4c6-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:16.676132 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.676135    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tns2c\" (UniqueName: \"kubernetes.io/projected/52bf2207-e86a-4f23-b423-10e83de2b4c6-kube-api-access-tns2c\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:16.954496 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.954459    2562 generic.go:356] "Generic (PLEG): container finished" podID="52bf2207-e86a-4f23-b423-10e83de2b4c6" containerID="2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816" exitCode=0
Jul 15 20:45:16.954677 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.954505    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"
Jul 15 20:45:16.954677 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.954543    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c" event={"ID":"52bf2207-e86a-4f23-b423-10e83de2b4c6","Type":"ContainerDied","Data":"2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816"}
Jul 15 20:45:16.954677 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.954586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c" event={"ID":"52bf2207-e86a-4f23-b423-10e83de2b4c6","Type":"ContainerDied","Data":"03435906be4c7ee8900f4f78e9a0ec0fde7d575e35948af9021781c3887ddda3"}
Jul 15 20:45:16.954677 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.954608    2562 scope.go:117] "RemoveContainer" containerID="2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816"
Jul 15 20:45:16.963824 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.963805    2562 scope.go:117] "RemoveContainer" containerID="2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816"
Jul 15 20:45:16.964117 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:45:16.964097    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816\": container with ID starting with 2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816 not found: ID does not exist" containerID="2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816"
Jul 15 20:45:16.964178 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.964125    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816"} err="failed to get container status \"2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816\": rpc error: code = NotFound desc = could not find container \"2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816\": container with ID starting with 2589090994a36351d7484611c8df7ad0fc840ac26e54529e12da3bffa59cc816 not found: ID does not exist"
Jul 15 20:45:16.980896 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.978441    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"]
Jul 15 20:45:16.983317 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:16.983290    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-724347/sub-test-model-724347-kserve-6c9bdcbbcc-hrk7c"]
Jul 15 20:45:17.078515 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.078483    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52bf2207-e86a-4f23-b423-10e83de2b4c6" path="/var/lib/kubelet/pods/52bf2207-e86a-4f23-b423-10e83de2b4c6/volumes"
Jul 15 20:45:17.255213 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.255119    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm" podUID="6f5db649-f0e1-4997-bab7-36791125a75e" containerName="main" containerID="cri-o://71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf" gracePeriod=45
Jul 15 20:45:17.509146 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.509072    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:45:17.588098 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588068    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9spc\" (UniqueName: \"kubernetes.io/projected/6f5db649-f0e1-4997-bab7-36791125a75e-kube-api-access-g9spc\") pod \"6f5db649-f0e1-4997-bab7-36791125a75e\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") "
Jul 15 20:45:17.588098 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588107    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-home\") pod \"6f5db649-f0e1-4997-bab7-36791125a75e\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") "
Jul 15 20:45:17.588347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588161    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-tmp-dir\") pod \"6f5db649-f0e1-4997-bab7-36791125a75e\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") "
Jul 15 20:45:17.588347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588220    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-dshm\") pod \"6f5db649-f0e1-4997-bab7-36791125a75e\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") "
Jul 15 20:45:17.588347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588258    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5db649-f0e1-4997-bab7-36791125a75e-tls-certs\") pod \"6f5db649-f0e1-4997-bab7-36791125a75e\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") "
Jul 15 20:45:17.588347 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588297    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-model-cache\") pod \"6f5db649-f0e1-4997-bab7-36791125a75e\" (UID: \"6f5db649-f0e1-4997-bab7-36791125a75e\") "
Jul 15 20:45:17.588532 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588399    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-home" (OuterVolumeSpecName: "home") pod "6f5db649-f0e1-4997-bab7-36791125a75e" (UID: "6f5db649-f0e1-4997-bab7-36791125a75e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:17.588532 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588410    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f5db649-f0e1-4997-bab7-36791125a75e" (UID: "6f5db649-f0e1-4997-bab7-36791125a75e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:17.588626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588557    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f5db649-f0e1-4997-bab7-36791125a75e" (UID: "6f5db649-f0e1-4997-bab7-36791125a75e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:17.588626 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588610    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:17.588722 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.588635    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:17.590201 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.590175    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f5db649-f0e1-4997-bab7-36791125a75e-kube-api-access-g9spc" (OuterVolumeSpecName: "kube-api-access-g9spc") pod "6f5db649-f0e1-4997-bab7-36791125a75e" (UID: "6f5db649-f0e1-4997-bab7-36791125a75e"). InnerVolumeSpecName "kube-api-access-g9spc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:45:17.590330 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.590239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-dshm" (OuterVolumeSpecName: "dshm") pod "6f5db649-f0e1-4997-bab7-36791125a75e" (UID: "6f5db649-f0e1-4997-bab7-36791125a75e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:45:17.590384 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.590347    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f5db649-f0e1-4997-bab7-36791125a75e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f5db649-f0e1-4997-bab7-36791125a75e" (UID: "6f5db649-f0e1-4997-bab7-36791125a75e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:45:17.689983 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.689940    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:17.690148 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.690004    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9spc\" (UniqueName: \"kubernetes.io/projected/6f5db649-f0e1-4997-bab7-36791125a75e-kube-api-access-g9spc\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:17.690148 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.690014    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f5db649-f0e1-4997-bab7-36791125a75e-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:17.690148 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.690023    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5db649-f0e1-4997-bab7-36791125a75e-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:45:17.961772 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.961735    2562 generic.go:356] "Generic (PLEG): container finished" podID="6f5db649-f0e1-4997-bab7-36791125a75e" containerID="71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf" exitCode=0
Jul 15 20:45:17.961947 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.961781    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"
Jul 15 20:45:17.961947 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.961778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm" event={"ID":"6f5db649-f0e1-4997-bab7-36791125a75e","Type":"ContainerDied","Data":"71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf"}
Jul 15 20:45:17.961947 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.961894    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm" event={"ID":"6f5db649-f0e1-4997-bab7-36791125a75e","Type":"ContainerDied","Data":"b20216bcda3a7ea3c1136f556a8f22f5f3d72a5415fc255fb8b8be84ac334891"}
Jul 15 20:45:17.961947 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.961919    2562 scope.go:117] "RemoveContainer" containerID="71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf"
Jul 15 20:45:17.971333 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.971308    2562 scope.go:117] "RemoveContainer" containerID="71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf"
Jul 15 20:45:17.971636 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:45:17.971617    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf\": container with ID starting with 71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf not found: ID does not exist" containerID="71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf"
Jul 15 20:45:17.971681 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.971646    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf"} err="failed to get container status \"71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf\": rpc error: code = NotFound desc = could not find container \"71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf\": container with ID starting with 71b64ff73f7bb625d9d7cbd84ed3a9348aa19192b8c8fda1a496500e5daa6cdf not found: ID does not exist"
Jul 15 20:45:17.982745 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.982716    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"]
Jul 15 20:45:17.986720 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:17.986686    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/sub-test-model-5292f5-kserve-5c676d447c-jb8wm"]
Jul 15 20:45:19.026522 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.026489    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"]
Jul 15 20:45:19.027144 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.027124    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5db649-f0e1-4997-bab7-36791125a75e" containerName="main"
Jul 15 20:45:19.027195 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.027149    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5db649-f0e1-4997-bab7-36791125a75e" containerName="main"
Jul 15 20:45:19.027195 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.027181    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bf2207-e86a-4f23-b423-10e83de2b4c6" containerName="main"
Jul 15 20:45:19.027195 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.027190    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bf2207-e86a-4f23-b423-10e83de2b4c6" containerName="main"
Jul 15 20:45:19.027289 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.027275    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bf2207-e86a-4f23-b423-10e83de2b4c6" containerName="main"
Jul 15 20:45:19.027323 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.027290    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5db649-f0e1-4997-bab7-36791125a75e" containerName="main"
Jul 15 20:45:19.032080 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.032058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.034947 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.034888    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"openshift-service-ca.crt\""
Jul 15 20:45:19.035102 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.034992    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"kube-root-ca.crt\""
Jul 15 20:45:19.036449 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.036425    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"default-dockercfg-8s9fq\""
Jul 15 20:45:19.036449 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.036437    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-724347\"/\"rate-test-model-724347-kserve-self-signed-certs\""
Jul 15 20:45:19.038952 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.038932    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"]
Jul 15 20:45:19.079791 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.079761    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f5db649-f0e1-4997-bab7-36791125a75e" path="/var/lib/kubelet/pods/6f5db649-f0e1-4997-bab7-36791125a75e/volumes"
Jul 15 20:45:19.100918 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.100892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-model-cache\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.101082 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.100932    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-dshm\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.101082 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.101011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-home\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.101082 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.101037    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-tmp-dir\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.101177 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.101107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/4f296596-5715-40f0-9e57-d93152dcb7e9-kube-api-access-v7jtq\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.101177 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.101129    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f296596-5715-40f0-9e57-d93152dcb7e9-tls-certs\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.202327 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202295    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-model-cache\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.202494 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-dshm\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.202494 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-home\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.202494 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-tmp-dir\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.202494 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/4f296596-5715-40f0-9e57-d93152dcb7e9-kube-api-access-v7jtq\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.202494 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f296596-5715-40f0-9e57-d93152dcb7e9-tls-certs\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.202910 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-home\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.203074 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202920    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-tmp-dir\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.203074 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.202963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-model-cache\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.204754 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.204728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-dshm\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.204842 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.204824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f296596-5715-40f0-9e57-d93152dcb7e9-tls-certs\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.214186 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.214156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/4f296596-5715-40f0-9e57-d93152dcb7e9-kube-api-access-v7jtq\") pod \"rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") " pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.354287 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.354200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:19.489070 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.489050    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"]
Jul 15 20:45:19.491347 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:45:19.491321    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f296596_5715_40f0_9e57_d93152dcb7e9.slice/crio-176e04e1b76e423f03c31614cfe73d140e075e5c989e6b9a5379d237bf14eb21 WatchSource:0}: Error finding container 176e04e1b76e423f03c31614cfe73d140e075e5c989e6b9a5379d237bf14eb21: Status 404 returned error can't find the container with id 176e04e1b76e423f03c31614cfe73d140e075e5c989e6b9a5379d237bf14eb21
Jul 15 20:45:19.972369 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.972332    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7" event={"ID":"4f296596-5715-40f0-9e57-d93152dcb7e9","Type":"ContainerStarted","Data":"ade2931173ae7244490350ea53777e4a878e86bf26588a9684ebb5d31815f34f"}
Jul 15 20:45:19.972369 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.972378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7" event={"ID":"4f296596-5715-40f0-9e57-d93152dcb7e9","Type":"ContainerStarted","Data":"176e04e1b76e423f03c31614cfe73d140e075e5c989e6b9a5379d237bf14eb21"}
Jul 15 20:45:19.992718 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:19.992668    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7" podStartSLOduration=0.992653868 podStartE2EDuration="992.653868ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:45:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:45:19.990382938 +0000 UTC m=+2801.438756032" watchObservedRunningTime="2026-07-15 20:45:19.992653868 +0000 UTC m=+2801.441026977"
Jul 15 20:45:29.355330 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:29.355295    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:29.355737 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:29.355355    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:29.367649 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:29.367623    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:45:30.029090 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:45:30.029061    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:46:00.947925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.947892    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"]
Jul 15 20:46:00.951025 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.951003    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:00.953591 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.953572    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"kube-root-ca.crt\""
Jul 15 20:46:00.954663 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.954638    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"rate-test-model-5292f5-kserve-self-signed-certs\""
Jul 15 20:46:00.954782 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.954685    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"default-dockercfg-n8667\""
Jul 15 20:46:00.954782 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.954729    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5292f5\"/\"openshift-service-ca.crt\""
Jul 15 20:46:00.965019 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.964991    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"]
Jul 15 20:46:00.988314 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.988289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-home\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:00.988458 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.988339    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tls-certs\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:00.988458 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.988439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-kube-api-access-9b9kt\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:00.988563 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.988473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tmp-dir\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:00.988563 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.988511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-dshm\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:00.988673 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:00.988566    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-model-cache\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089113 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-kube-api-access-9b9kt\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089285 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tmp-dir\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089285 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-dshm\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089285 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089180    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-model-cache\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089285 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-home\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089285 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tls-certs\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089556 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089531    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tmp-dir\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089612 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-home\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.089685 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.089661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-model-cache\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.091533 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.091506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-dshm\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.091636 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.091588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tls-certs\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.097330 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.097309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-kube-api-access-9b9kt\") pod \"rate-test-model-5292f5-kserve-f45c98d6c-cfbdx\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") " pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.259764 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.259680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:01.391787 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:01.391758    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"]
Jul 15 20:46:01.394041 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:46:01.394006    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fb0aa48_f59a_4655_b1bc_6c5df7c847c8.slice/crio-37a8a2f6369b29afb2d9929f85441188e1b82b5ead5b6eedc02eb1315faeb68e WatchSource:0}: Error finding container 37a8a2f6369b29afb2d9929f85441188e1b82b5ead5b6eedc02eb1315faeb68e: Status 404 returned error can't find the container with id 37a8a2f6369b29afb2d9929f85441188e1b82b5ead5b6eedc02eb1315faeb68e
Jul 15 20:46:02.172557 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:02.172516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx" event={"ID":"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8","Type":"ContainerStarted","Data":"4b487869698341a4b1cea2074131b9d9f90384a5146814ef78a404e9d980c0a7"}
Jul 15 20:46:02.173044 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:02.172566    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx" event={"ID":"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8","Type":"ContainerStarted","Data":"37a8a2f6369b29afb2d9929f85441188e1b82b5ead5b6eedc02eb1315faeb68e"}
Jul 15 20:46:02.191121 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:02.191068    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx" podStartSLOduration=2.191049889 podStartE2EDuration="2.191049889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:46:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:46:02.190137185 +0000 UTC m=+2843.638510284" watchObservedRunningTime="2026-07-15 20:46:02.191049889 +0000 UTC m=+2843.639422985"
Jul 15 20:46:11.260346 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:11.260309    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:11.260841 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:11.260559    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:11.272996 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:11.272954    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:12.231902 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:12.231873    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:46:49.325206 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:49.325173    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"]
Jul 15 20:46:50.357849 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:50.357818    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"]
Jul 15 20:46:51.255272 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.255234    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7592-140704134937008"]
Jul 15 20:46:51.257765 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.257745    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134937008"
Jul 15 20:46:51.260426 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.260403    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8knck\""
Jul 15 20:46:51.264151 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.264121    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134937008"]
Jul 15 20:46:51.367108 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.367074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c54p7\" (UniqueName: \"kubernetes.io/projected/0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799-kube-api-access-c54p7\") pod \"test-curl-7592-140704134937008\" (UID: \"0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799\") " pod="opendatahub/test-curl-7592-140704134937008"
Jul 15 20:46:51.467899 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.467861    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c54p7\" (UniqueName: \"kubernetes.io/projected/0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799-kube-api-access-c54p7\") pod \"test-curl-7592-140704134937008\" (UID: \"0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799\") " pod="opendatahub/test-curl-7592-140704134937008"
Jul 15 20:46:51.476807 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.476771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c54p7\" (UniqueName: \"kubernetes.io/projected/0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799-kube-api-access-c54p7\") pod \"test-curl-7592-140704134937008\" (UID: \"0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799\") " pod="opendatahub/test-curl-7592-140704134937008"
Jul 15 20:46:51.568614 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.568501    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134937008"
Jul 15 20:46:51.704125 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.704099    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134937008"]
Jul 15 20:46:51.705329 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:46:51.705301    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fa0c0a4_65b8_4e9e_b6e6_6d4b59224799.slice/crio-75895356064bce01aaeae94d01f44b7a26a40e8eb2ec28893df9ba2ca3de3fa6 WatchSource:0}: Error finding container 75895356064bce01aaeae94d01f44b7a26a40e8eb2ec28893df9ba2ca3de3fa6: Status 404 returned error can't find the container with id 75895356064bce01aaeae94d01f44b7a26a40e8eb2ec28893df9ba2ca3de3fa6
Jul 15 20:46:51.706618 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:51.706597    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:46:52.413555 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:52.413528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134937008" event={"ID":"0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799","Type":"ContainerStarted","Data":"75895356064bce01aaeae94d01f44b7a26a40e8eb2ec28893df9ba2ca3de3fa6"}
Jul 15 20:46:53.419594 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:53.419557    2562 generic.go:356] "Generic (PLEG): container finished" podID="0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799" containerID="d2c668a7ccf1c6ad2da411782e40953830ce3334c91f24de309ce02b67047a21" exitCode=0
Jul 15 20:46:53.419992 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:53.419642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134937008" event={"ID":"0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799","Type":"ContainerDied","Data":"d2c668a7ccf1c6ad2da411782e40953830ce3334c91f24de309ce02b67047a21"}
Jul 15 20:46:54.558048 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.558017    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134937008"
Jul 15 20:46:54.595687 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.595659    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7592-140704134937008_0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799/test-curl-7592-140704134937008/0.log"
Jul 15 20:46:54.618356 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.618322    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7592-140704134937008"]
Jul 15 20:46:54.621954 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.621929    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7592-140704134937008"]
Jul 15 20:46:54.697795 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.697766    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c54p7\" (UniqueName: \"kubernetes.io/projected/0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799-kube-api-access-c54p7\") pod \"0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799\" (UID: \"0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799\") "
Jul 15 20:46:54.699825 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.699802    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799-kube-api-access-c54p7" (OuterVolumeSpecName: "kube-api-access-c54p7") pod "0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799" (UID: "0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799"). InnerVolumeSpecName "kube-api-access-c54p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:46:54.739094 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.739062    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7592-140704134935344"]
Jul 15 20:46:54.739489 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.739477    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799" containerName="test-curl-7592-140704134937008"
Jul 15 20:46:54.739535 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.739491    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799" containerName="test-curl-7592-140704134937008"
Jul 15 20:46:54.739570 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.739552    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799" containerName="test-curl-7592-140704134937008"
Jul 15 20:46:54.742625 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.742609    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:54.749131 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.749108    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134935344"]
Jul 15 20:46:54.799333 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.799302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g87jq\" (UniqueName: \"kubernetes.io/projected/6fa101c1-cdca-4725-bb62-cccf5778822c-kube-api-access-g87jq\") pod \"test-curl-7592-140704134935344\" (UID: \"6fa101c1-cdca-4725-bb62-cccf5778822c\") " pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:54.799480 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.799423    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c54p7\" (UniqueName: \"kubernetes.io/projected/0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799-kube-api-access-c54p7\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:46:54.900804 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.900765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g87jq\" (UniqueName: \"kubernetes.io/projected/6fa101c1-cdca-4725-bb62-cccf5778822c-kube-api-access-g87jq\") pod \"test-curl-7592-140704134935344\" (UID: \"6fa101c1-cdca-4725-bb62-cccf5778822c\") " pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:54.910268 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:54.910234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g87jq\" (UniqueName: \"kubernetes.io/projected/6fa101c1-cdca-4725-bb62-cccf5778822c-kube-api-access-g87jq\") pod \"test-curl-7592-140704134935344\" (UID: \"6fa101c1-cdca-4725-bb62-cccf5778822c\") " pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:55.052598 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:55.052513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:55.080752 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:55.080718    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799" path="/var/lib/kubelet/pods/0fa0c0a4-65b8-4e9e-b6e6-6d4b59224799/volumes"
Jul 15 20:46:55.188709 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:55.188683    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134935344"]
Jul 15 20:46:55.190090 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:46:55.190068    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fa101c1_cdca_4725_bb62_cccf5778822c.slice/crio-bc9f0226e511b6d27fa55bfab6e811091ee333b2f43ebd05f097514409a98a96 WatchSource:0}: Error finding container bc9f0226e511b6d27fa55bfab6e811091ee333b2f43ebd05f097514409a98a96: Status 404 returned error can't find the container with id bc9f0226e511b6d27fa55bfab6e811091ee333b2f43ebd05f097514409a98a96
Jul 15 20:46:55.431317 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:55.431224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134935344" event={"ID":"6fa101c1-cdca-4725-bb62-cccf5778822c","Type":"ContainerStarted","Data":"bc9f0226e511b6d27fa55bfab6e811091ee333b2f43ebd05f097514409a98a96"}
Jul 15 20:46:55.432336 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:55.432317    2562 scope.go:117] "RemoveContainer" containerID="d2c668a7ccf1c6ad2da411782e40953830ce3334c91f24de309ce02b67047a21"
Jul 15 20:46:55.432456 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:55.432331    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134937008"
Jul 15 20:46:56.439848 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:56.439812    2562 generic.go:356] "Generic (PLEG): container finished" podID="6fa101c1-cdca-4725-bb62-cccf5778822c" containerID="b521128505279b67286e921f45bae790522fa9a646a0e5b2f97f71a30144b089" exitCode=0
Jul 15 20:46:56.440298 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:56.439892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134935344" event={"ID":"6fa101c1-cdca-4725-bb62-cccf5778822c","Type":"ContainerDied","Data":"b521128505279b67286e921f45bae790522fa9a646a0e5b2f97f71a30144b089"}
Jul 15 20:46:57.583019 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.582996    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:57.604610 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.604585    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7592-140704134935344_6fa101c1-cdca-4725-bb62-cccf5778822c/test-curl-7592-140704134935344/0.log"
Jul 15 20:46:57.628188 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.628139    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7592-140704134935344"]
Jul 15 20:46:57.631100 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.631073    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7592-140704134935344"]
Jul 15 20:46:57.728827 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.728740    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g87jq\" (UniqueName: \"kubernetes.io/projected/6fa101c1-cdca-4725-bb62-cccf5778822c-kube-api-access-g87jq\") pod \"6fa101c1-cdca-4725-bb62-cccf5778822c\" (UID: \"6fa101c1-cdca-4725-bb62-cccf5778822c\") "
Jul 15 20:46:57.730836 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.730814    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fa101c1-cdca-4725-bb62-cccf5778822c-kube-api-access-g87jq" (OuterVolumeSpecName: "kube-api-access-g87jq") pod "6fa101c1-cdca-4725-bb62-cccf5778822c" (UID: "6fa101c1-cdca-4725-bb62-cccf5778822c"). InnerVolumeSpecName "kube-api-access-g87jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:46:57.795668 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.795637    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:46:57.796032 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.796018    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa101c1-cdca-4725-bb62-cccf5778822c" containerName="test-curl-7592-140704134935344"
Jul 15 20:46:57.796084 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.796034    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa101c1-cdca-4725-bb62-cccf5778822c" containerName="test-curl-7592-140704134935344"
Jul 15 20:46:57.796120 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.796109    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa101c1-cdca-4725-bb62-cccf5778822c" containerName="test-curl-7592-140704134935344"
Jul 15 20:46:57.803990 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.803934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:46:57.807689 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.807662    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:46:57.829837 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.829802    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g87jq\" (UniqueName: \"kubernetes.io/projected/6fa101c1-cdca-4725-bb62-cccf5778822c-kube-api-access-g87jq\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:46:57.931239 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:57.931193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpzch\" (UniqueName: \"kubernetes.io/projected/a7ea0d64-b477-4a97-9628-740d62b71c2f-kube-api-access-vpzch\") pod \"test-curl-7592-140704134830672\" (UID: \"a7ea0d64-b477-4a97-9628-740d62b71c2f\") " pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:46:58.032340 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:58.032250    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpzch\" (UniqueName: \"kubernetes.io/projected/a7ea0d64-b477-4a97-9628-740d62b71c2f-kube-api-access-vpzch\") pod \"test-curl-7592-140704134830672\" (UID: \"a7ea0d64-b477-4a97-9628-740d62b71c2f\") " pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:46:58.040763 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:58.040736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpzch\" (UniqueName: \"kubernetes.io/projected/a7ea0d64-b477-4a97-9628-740d62b71c2f-kube-api-access-vpzch\") pod \"test-curl-7592-140704134830672\" (UID: \"a7ea0d64-b477-4a97-9628-740d62b71c2f\") " pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:46:58.115239 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:58.115202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:46:58.252649 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:58.252610    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:46:58.255696 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:46:58.255670    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7ea0d64_b477_4a97_9628_740d62b71c2f.slice/crio-99702ac9af7b59a934c6e550390c768c7bc2fb2422fee283125664809a735d66 WatchSource:0}: Error finding container 99702ac9af7b59a934c6e550390c768c7bc2fb2422fee283125664809a735d66: Status 404 returned error can't find the container with id 99702ac9af7b59a934c6e550390c768c7bc2fb2422fee283125664809a735d66
Jul 15 20:46:58.451558 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:58.451526    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc9f0226e511b6d27fa55bfab6e811091ee333b2f43ebd05f097514409a98a96"
Jul 15 20:46:58.451558 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:58.451537    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:58.452793 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:58.452763    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134830672" event={"ID":"a7ea0d64-b477-4a97-9628-740d62b71c2f","Type":"ContainerStarted","Data":"99702ac9af7b59a934c6e550390c768c7bc2fb2422fee283125664809a735d66"}
Jul 15 20:46:58.462036 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:46:58.462006    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7592-140704134935344\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="6fa101c1-cdca-4725-bb62-cccf5778822c" pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:59.078498 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:46:59.078455    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7592-140704134935344\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="6fa101c1-cdca-4725-bb62-cccf5778822c" pod="opendatahub/test-curl-7592-140704134935344"
Jul 15 20:46:59.079450 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:59.079429    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fa101c1-cdca-4725-bb62-cccf5778822c" path="/var/lib/kubelet/pods/6fa101c1-cdca-4725-bb62-cccf5778822c/volumes"
Jul 15 20:46:59.459824 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:59.459786    2562 generic.go:356] "Generic (PLEG): container finished" podID="a7ea0d64-b477-4a97-9628-740d62b71c2f" containerID="c596aa9227f9a19be5525a44149e012764717d634ecaf414994ab62a0c10fe1c" exitCode=0
Jul 15 20:46:59.460028 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:46:59.459836    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134830672" event={"ID":"a7ea0d64-b477-4a97-9628-740d62b71c2f","Type":"ContainerDied","Data":"c596aa9227f9a19be5525a44149e012764717d634ecaf414994ab62a0c10fe1c"}
Jul 15 20:47:00.603332 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.603306    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:00.625091 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.625060    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7592-140704134830672_a7ea0d64-b477-4a97-9628-740d62b71c2f/test-curl-7592-140704134830672/0.log"
Jul 15 20:47:00.651690 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.651663    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:47:00.655519 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.655491    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:47:00.759003 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.758885    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpzch\" (UniqueName: \"kubernetes.io/projected/a7ea0d64-b477-4a97-9628-740d62b71c2f-kube-api-access-vpzch\") pod \"a7ea0d64-b477-4a97-9628-740d62b71c2f\" (UID: \"a7ea0d64-b477-4a97-9628-740d62b71c2f\") "
Jul 15 20:47:00.761089 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.761063    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ea0d64-b477-4a97-9628-740d62b71c2f-kube-api-access-vpzch" (OuterVolumeSpecName: "kube-api-access-vpzch") pod "a7ea0d64-b477-4a97-9628-740d62b71c2f" (UID: "a7ea0d64-b477-4a97-9628-740d62b71c2f"). InnerVolumeSpecName "kube-api-access-vpzch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:47:00.780087 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.780054    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:47:00.780454 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.780440    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ea0d64-b477-4a97-9628-740d62b71c2f" containerName="test-curl-7592-140704134830672"
Jul 15 20:47:00.780499 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.780455    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ea0d64-b477-4a97-9628-740d62b71c2f" containerName="test-curl-7592-140704134830672"
Jul 15 20:47:00.780534 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.780522    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ea0d64-b477-4a97-9628-740d62b71c2f" containerName="test-curl-7592-140704134830672"
Jul 15 20:47:00.783516 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.783497    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:00.788924 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.788898    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:47:00.859676 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.859634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpj94\" (UniqueName: \"kubernetes.io/projected/0a22190f-14b1-4b7d-a77d-57f154d3b5b3-kube-api-access-vpj94\") pod \"test-curl-7592-140704134830672\" (UID: \"0a22190f-14b1-4b7d-a77d-57f154d3b5b3\") " pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:00.859886 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.859795    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpzch\" (UniqueName: \"kubernetes.io/projected/a7ea0d64-b477-4a97-9628-740d62b71c2f-kube-api-access-vpzch\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:00.960761 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.960720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj94\" (UniqueName: \"kubernetes.io/projected/0a22190f-14b1-4b7d-a77d-57f154d3b5b3-kube-api-access-vpj94\") pod \"test-curl-7592-140704134830672\" (UID: \"0a22190f-14b1-4b7d-a77d-57f154d3b5b3\") " pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:00.969543 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:00.969506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpj94\" (UniqueName: \"kubernetes.io/projected/0a22190f-14b1-4b7d-a77d-57f154d3b5b3-kube-api-access-vpj94\") pod \"test-curl-7592-140704134830672\" (UID: \"0a22190f-14b1-4b7d-a77d-57f154d3b5b3\") " pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:01.079659 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:01.079578    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ea0d64-b477-4a97-9628-740d62b71c2f" path="/var/lib/kubelet/pods/a7ea0d64-b477-4a97-9628-740d62b71c2f/volumes"
Jul 15 20:47:01.094027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:01.093996    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:01.234242 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:01.234213    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:47:01.236630 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:47:01.236604    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a22190f_14b1_4b7d_a77d_57f154d3b5b3.slice/crio-8c2f16f5efad58658e32d926d2fe9b2f6a0b542fd7e51a58dfd0228b5ebdceaf WatchSource:0}: Error finding container 8c2f16f5efad58658e32d926d2fe9b2f6a0b542fd7e51a58dfd0228b5ebdceaf: Status 404 returned error can't find the container with id 8c2f16f5efad58658e32d926d2fe9b2f6a0b542fd7e51a58dfd0228b5ebdceaf
Jul 15 20:47:01.471152 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:01.471120    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134830672" event={"ID":"0a22190f-14b1-4b7d-a77d-57f154d3b5b3","Type":"ContainerStarted","Data":"8c2f16f5efad58658e32d926d2fe9b2f6a0b542fd7e51a58dfd0228b5ebdceaf"}
Jul 15 20:47:01.472277 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:01.472254    2562 scope.go:117] "RemoveContainer" containerID="c596aa9227f9a19be5525a44149e012764717d634ecaf414994ab62a0c10fe1c"
Jul 15 20:47:01.472277 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:01.472267    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:02.478963 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:02.478924    2562 generic.go:356] "Generic (PLEG): container finished" podID="0a22190f-14b1-4b7d-a77d-57f154d3b5b3" containerID="e2865f5e63e4e70fc33e9de2c49490780b3fbf5a78c8a429248c5a5d1b72e553" exitCode=0
Jul 15 20:47:02.479400 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:02.479018    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134830672" event={"ID":"0a22190f-14b1-4b7d-a77d-57f154d3b5b3","Type":"ContainerDied","Data":"e2865f5e63e4e70fc33e9de2c49490780b3fbf5a78c8a429248c5a5d1b72e553"}
Jul 15 20:47:03.622264 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.622238    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:03.643882 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.643843    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7592-140704134830672_0a22190f-14b1-4b7d-a77d-57f154d3b5b3/test-curl-7592-140704134830672/0.log"
Jul 15 20:47:03.668075 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.668042    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:47:03.672822 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.672795    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7592-140704134830672"]
Jul 15 20:47:03.790266 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.790183    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpj94\" (UniqueName: \"kubernetes.io/projected/0a22190f-14b1-4b7d-a77d-57f154d3b5b3-kube-api-access-vpj94\") pod \"0a22190f-14b1-4b7d-a77d-57f154d3b5b3\" (UID: \"0a22190f-14b1-4b7d-a77d-57f154d3b5b3\") "
Jul 15 20:47:03.792327 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.792281    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a22190f-14b1-4b7d-a77d-57f154d3b5b3-kube-api-access-vpj94" (OuterVolumeSpecName: "kube-api-access-vpj94") pod "0a22190f-14b1-4b7d-a77d-57f154d3b5b3" (UID: "0a22190f-14b1-4b7d-a77d-57f154d3b5b3"). InnerVolumeSpecName "kube-api-access-vpj94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:47:03.794709 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.794686    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:03.795090 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.795075    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a22190f-14b1-4b7d-a77d-57f154d3b5b3" containerName="test-curl-7592-140704134830672"
Jul 15 20:47:03.795141 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.795091    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a22190f-14b1-4b7d-a77d-57f154d3b5b3" containerName="test-curl-7592-140704134830672"
Jul 15 20:47:03.795178 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.795173    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a22190f-14b1-4b7d-a77d-57f154d3b5b3" containerName="test-curl-7592-140704134830672"
Jul 15 20:47:03.798232 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.798178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:03.802734 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.802705    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:03.891018 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.890963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x58dk\" (UniqueName: \"kubernetes.io/projected/398916fa-ea54-4bba-8b15-87925384c894-kube-api-access-x58dk\") pod \"test-curl-7592-140704134831680\" (UID: \"398916fa-ea54-4bba-8b15-87925384c894\") " pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:03.891205 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.891101    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpj94\" (UniqueName: \"kubernetes.io/projected/0a22190f-14b1-4b7d-a77d-57f154d3b5b3-kube-api-access-vpj94\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:03.992060 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:03.992024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x58dk\" (UniqueName: \"kubernetes.io/projected/398916fa-ea54-4bba-8b15-87925384c894-kube-api-access-x58dk\") pod \"test-curl-7592-140704134831680\" (UID: \"398916fa-ea54-4bba-8b15-87925384c894\") " pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:04.000561 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.000532    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x58dk\" (UniqueName: \"kubernetes.io/projected/398916fa-ea54-4bba-8b15-87925384c894-kube-api-access-x58dk\") pod \"test-curl-7592-140704134831680\" (UID: \"398916fa-ea54-4bba-8b15-87925384c894\") " pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:04.109065 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.108954    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:04.247928 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.247898    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:04.250400 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:47:04.250375    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod398916fa_ea54_4bba_8b15_87925384c894.slice/crio-cbb05a56d2f1b7efe8610fc6601264313731ba4cad4f47d21a27acc17a1c9aa4 WatchSource:0}: Error finding container cbb05a56d2f1b7efe8610fc6601264313731ba4cad4f47d21a27acc17a1c9aa4: Status 404 returned error can't find the container with id cbb05a56d2f1b7efe8610fc6601264313731ba4cad4f47d21a27acc17a1c9aa4
Jul 15 20:47:04.347842 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.347794    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7" podUID="4f296596-5715-40f0-9e57-d93152dcb7e9" containerName="main" containerID="cri-o://ade2931173ae7244490350ea53777e4a878e86bf26588a9684ebb5d31815f34f" gracePeriod=45
Jul 15 20:47:04.492184 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.492144    2562 generic.go:356] "Generic (PLEG): container finished" podID="4f296596-5715-40f0-9e57-d93152dcb7e9" containerID="ade2931173ae7244490350ea53777e4a878e86bf26588a9684ebb5d31815f34f" exitCode=0
Jul 15 20:47:04.492333 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.492207    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7" event={"ID":"4f296596-5715-40f0-9e57-d93152dcb7e9","Type":"ContainerDied","Data":"ade2931173ae7244490350ea53777e4a878e86bf26588a9684ebb5d31815f34f"}
Jul 15 20:47:04.493671 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.493651    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:04.493671 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.493666    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c2f16f5efad58658e32d926d2fe9b2f6a0b542fd7e51a58dfd0228b5ebdceaf"
Jul 15 20:47:04.494931 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.494900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134831680" event={"ID":"398916fa-ea54-4bba-8b15-87925384c894","Type":"ContainerStarted","Data":"cbb05a56d2f1b7efe8610fc6601264313731ba4cad4f47d21a27acc17a1c9aa4"}
Jul 15 20:47:04.504872 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:47:04.504840    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7592-140704134830672\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="0a22190f-14b1-4b7d-a77d-57f154d3b5b3" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:04.592248 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.592223    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:47:04.594390 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:47:04.594363    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7592-140704134830672\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="0a22190f-14b1-4b7d-a77d-57f154d3b5b3" pod="opendatahub/test-curl-7592-140704134830672"
Jul 15 20:47:04.698447 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698415    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-home\") pod \"4f296596-5715-40f0-9e57-d93152dcb7e9\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") "
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698484    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-dshm\") pod \"4f296596-5715-40f0-9e57-d93152dcb7e9\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") "
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698521    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/4f296596-5715-40f0-9e57-d93152dcb7e9-kube-api-access-v7jtq\") pod \"4f296596-5715-40f0-9e57-d93152dcb7e9\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") "
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698572    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-tmp-dir\") pod \"4f296596-5715-40f0-9e57-d93152dcb7e9\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") "
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-model-cache\") pod \"4f296596-5715-40f0-9e57-d93152dcb7e9\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") "
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698630    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f296596-5715-40f0-9e57-d93152dcb7e9-tls-certs\") pod \"4f296596-5715-40f0-9e57-d93152dcb7e9\" (UID: \"4f296596-5715-40f0-9e57-d93152dcb7e9\") "
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698730    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-home" (OuterVolumeSpecName: "home") pod "4f296596-5715-40f0-9e57-d93152dcb7e9" (UID: "4f296596-5715-40f0-9e57-d93152dcb7e9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698804    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f296596-5715-40f0-9e57-d93152dcb7e9" (UID: "4f296596-5715-40f0-9e57-d93152dcb7e9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:04.698925 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698883    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f296596-5715-40f0-9e57-d93152dcb7e9" (UID: "4f296596-5715-40f0-9e57-d93152dcb7e9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:04.699261 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.698961    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:04.699261 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.699005    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:04.699261 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.699014    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:04.700619 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.700598    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-dshm" (OuterVolumeSpecName: "dshm") pod "4f296596-5715-40f0-9e57-d93152dcb7e9" (UID: "4f296596-5715-40f0-9e57-d93152dcb7e9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:04.700724 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.700615    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f296596-5715-40f0-9e57-d93152dcb7e9-kube-api-access-v7jtq" (OuterVolumeSpecName: "kube-api-access-v7jtq") pod "4f296596-5715-40f0-9e57-d93152dcb7e9" (UID: "4f296596-5715-40f0-9e57-d93152dcb7e9"). InnerVolumeSpecName "kube-api-access-v7jtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:47:04.700724 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.700632    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f296596-5715-40f0-9e57-d93152dcb7e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f296596-5715-40f0-9e57-d93152dcb7e9" (UID: "4f296596-5715-40f0-9e57-d93152dcb7e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:47:04.800435 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.800323    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f296596-5715-40f0-9e57-d93152dcb7e9-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:04.800435 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.800363    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f296596-5715-40f0-9e57-d93152dcb7e9-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:04.800435 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:04.800377    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7jtq\" (UniqueName: \"kubernetes.io/projected/4f296596-5715-40f0-9e57-d93152dcb7e9-kube-api-access-v7jtq\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:05.079787 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.079695    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a22190f-14b1-4b7d-a77d-57f154d3b5b3" path="/var/lib/kubelet/pods/0a22190f-14b1-4b7d-a77d-57f154d3b5b3/volumes"
Jul 15 20:47:05.377672 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.377572    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx" podUID="8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" containerName="main" containerID="cri-o://4b487869698341a4b1cea2074131b9d9f90384a5146814ef78a404e9d980c0a7" gracePeriod=45
Jul 15 20:47:05.501345 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.501315    2562 generic.go:356] "Generic (PLEG): container finished" podID="8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" containerID="4b487869698341a4b1cea2074131b9d9f90384a5146814ef78a404e9d980c0a7" exitCode=0
Jul 15 20:47:05.501515 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.501385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx" event={"ID":"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8","Type":"ContainerDied","Data":"4b487869698341a4b1cea2074131b9d9f90384a5146814ef78a404e9d980c0a7"}
Jul 15 20:47:05.502933 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.502911    2562 generic.go:356] "Generic (PLEG): container finished" podID="398916fa-ea54-4bba-8b15-87925384c894" containerID="fa84c95d5dfdce7e28b720b8ed3768a5460a9d587d02e2aa223232a84da0e469" exitCode=0
Jul 15 20:47:05.503082 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.503005    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134831680" event={"ID":"398916fa-ea54-4bba-8b15-87925384c894","Type":"ContainerDied","Data":"fa84c95d5dfdce7e28b720b8ed3768a5460a9d587d02e2aa223232a84da0e469"}
Jul 15 20:47:05.504295 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.504276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7" event={"ID":"4f296596-5715-40f0-9e57-d93152dcb7e9","Type":"ContainerDied","Data":"176e04e1b76e423f03c31614cfe73d140e075e5c989e6b9a5379d237bf14eb21"}
Jul 15 20:47:05.504394 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.504307    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"
Jul 15 20:47:05.504394 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.504312    2562 scope.go:117] "RemoveContainer" containerID="ade2931173ae7244490350ea53777e4a878e86bf26588a9684ebb5d31815f34f"
Jul 15 20:47:05.538205 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.538169    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"]
Jul 15 20:47:05.543019 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.542985    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-724347/rate-test-model-724347-kserve-7cc9c9ddb7-lqsk7"]
Jul 15 20:47:05.648309 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.648280    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:47:05.807575 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.807538    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tmp-dir\") pod \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") "
Jul 15 20:47:05.808021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.807590    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tls-certs\") pod \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") "
Jul 15 20:47:05.808021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.807635    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-dshm\") pod \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") "
Jul 15 20:47:05.808021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.807650    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-model-cache\") pod \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") "
Jul 15 20:47:05.808021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.807722    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-home\") pod \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") "
Jul 15 20:47:05.808021 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.807752    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-kube-api-access-9b9kt\") pod \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\" (UID: \"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8\") "
Jul 15 20:47:05.808277 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.808089    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" (UID: "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:05.808339 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.808315    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-home" (OuterVolumeSpecName: "home") pod "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" (UID: "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:05.808388 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.808358    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" (UID: "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:05.808847 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.808825    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tmp-dir\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:05.808949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.808872    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-model-cache\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:05.808949 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.808888    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-home\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:05.812011 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.810597    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" (UID: "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:47:05.816896 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.816867    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-dshm" (OuterVolumeSpecName: "dshm") pod "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" (UID: "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:47:05.816896 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.816885    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-kube-api-access-9b9kt" (OuterVolumeSpecName: "kube-api-access-9b9kt") pod "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" (UID: "8fb0aa48-f59a-4655-b1bc-6c5df7c847c8"). InnerVolumeSpecName "kube-api-access-9b9kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:47:05.910351 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.910259    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-kube-api-access-9b9kt\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:05.910351 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.910292    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-tls-certs\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:05.910351 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:05.910304    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8-dshm\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:06.511461 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.511425    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx" event={"ID":"8fb0aa48-f59a-4655-b1bc-6c5df7c847c8","Type":"ContainerDied","Data":"37a8a2f6369b29afb2d9929f85441188e1b82b5ead5b6eedc02eb1315faeb68e"}
Jul 15 20:47:06.511461 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.511456    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"
Jul 15 20:47:06.511738 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.511481    2562 scope.go:117] "RemoveContainer" containerID="4b487869698341a4b1cea2074131b9d9f90384a5146814ef78a404e9d980c0a7"
Jul 15 20:47:06.533647 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.533607    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"]
Jul 15 20:47:06.538258 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.538234    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5292f5/rate-test-model-5292f5-kserve-f45c98d6c-cfbdx"]
Jul 15 20:47:06.650089 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.650063    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:06.677658 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.677633    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7592-140704134831680_398916fa-ea54-4bba-8b15-87925384c894/test-curl-7592-140704134831680/0.log"
Jul 15 20:47:06.704545 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.704513    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:06.711322 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.711295    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:06.817224 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.817134    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x58dk\" (UniqueName: \"kubernetes.io/projected/398916fa-ea54-4bba-8b15-87925384c894-kube-api-access-x58dk\") pod \"398916fa-ea54-4bba-8b15-87925384c894\" (UID: \"398916fa-ea54-4bba-8b15-87925384c894\") "
Jul 15 20:47:06.819258 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.819226    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/398916fa-ea54-4bba-8b15-87925384c894-kube-api-access-x58dk" (OuterVolumeSpecName: "kube-api-access-x58dk") pod "398916fa-ea54-4bba-8b15-87925384c894" (UID: "398916fa-ea54-4bba-8b15-87925384c894"). InnerVolumeSpecName "kube-api-access-x58dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:47:06.837695 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.837660    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:06.838097 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838083    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f296596-5715-40f0-9e57-d93152dcb7e9" containerName="main"
Jul 15 20:47:06.838145 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838099    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f296596-5715-40f0-9e57-d93152dcb7e9" containerName="main"
Jul 15 20:47:06.838145 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838110    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" containerName="main"
Jul 15 20:47:06.838145 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838117    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" containerName="main"
Jul 15 20:47:06.838145 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838128    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398916fa-ea54-4bba-8b15-87925384c894" containerName="test-curl-7592-140704134831680"
Jul 15 20:47:06.838145 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838134    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="398916fa-ea54-4bba-8b15-87925384c894" containerName="test-curl-7592-140704134831680"
Jul 15 20:47:06.838299 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838204    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f296596-5715-40f0-9e57-d93152dcb7e9" containerName="main"
Jul 15 20:47:06.838299 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838212    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="398916fa-ea54-4bba-8b15-87925384c894" containerName="test-curl-7592-140704134831680"
Jul 15 20:47:06.838299 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.838220    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" containerName="main"
Jul 15 20:47:06.841216 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.841199    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:06.852175 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.852151    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:06.918304 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:06.918269    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x58dk\" (UniqueName: \"kubernetes.io/projected/398916fa-ea54-4bba-8b15-87925384c894-kube-api-access-x58dk\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:07.019099 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.019053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxn6d\" (UniqueName: \"kubernetes.io/projected/e5efe276-6d9d-4f0f-9682-237754cac87c-kube-api-access-sxn6d\") pod \"test-curl-7592-140704134831680\" (UID: \"e5efe276-6d9d-4f0f-9682-237754cac87c\") " pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:07.078938 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.078858    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="398916fa-ea54-4bba-8b15-87925384c894" path="/var/lib/kubelet/pods/398916fa-ea54-4bba-8b15-87925384c894/volumes"
Jul 15 20:47:07.079253 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.079240    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f296596-5715-40f0-9e57-d93152dcb7e9" path="/var/lib/kubelet/pods/4f296596-5715-40f0-9e57-d93152dcb7e9/volumes"
Jul 15 20:47:07.079600 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.079588    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb0aa48-f59a-4655-b1bc-6c5df7c847c8" path="/var/lib/kubelet/pods/8fb0aa48-f59a-4655-b1bc-6c5df7c847c8/volumes"
Jul 15 20:47:07.120398 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.120352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxn6d\" (UniqueName: \"kubernetes.io/projected/e5efe276-6d9d-4f0f-9682-237754cac87c-kube-api-access-sxn6d\") pod \"test-curl-7592-140704134831680\" (UID: \"e5efe276-6d9d-4f0f-9682-237754cac87c\") " pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:07.130232 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.130205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxn6d\" (UniqueName: \"kubernetes.io/projected/e5efe276-6d9d-4f0f-9682-237754cac87c-kube-api-access-sxn6d\") pod \"test-curl-7592-140704134831680\" (UID: \"e5efe276-6d9d-4f0f-9682-237754cac87c\") " pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:07.150076 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.150044    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:07.292303 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.292280    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:07.294368 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:47:07.294341    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5efe276_6d9d_4f0f_9682_237754cac87c.slice/crio-361cc178c82d996c8091e3f4b3b2a8794507a89c382d1c5b0e1a95642646b3ee WatchSource:0}: Error finding container 361cc178c82d996c8091e3f4b3b2a8794507a89c382d1c5b0e1a95642646b3ee: Status 404 returned error can't find the container with id 361cc178c82d996c8091e3f4b3b2a8794507a89c382d1c5b0e1a95642646b3ee
Jul 15 20:47:07.516550 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.516513    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134831680" event={"ID":"e5efe276-6d9d-4f0f-9682-237754cac87c","Type":"ContainerStarted","Data":"361cc178c82d996c8091e3f4b3b2a8794507a89c382d1c5b0e1a95642646b3ee"}
Jul 15 20:47:07.518360 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.518339    2562 scope.go:117] "RemoveContainer" containerID="fa84c95d5dfdce7e28b720b8ed3768a5460a9d587d02e2aa223232a84da0e469"
Jul 15 20:47:07.518360 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:07.518358    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:08.525234 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:08.525199    2562 generic.go:356] "Generic (PLEG): container finished" podID="e5efe276-6d9d-4f0f-9682-237754cac87c" containerID="3ef056728f4642a5bcbc078d2787fdecb8de6bc0cfca542346ec5b4214673736" exitCode=0
Jul 15 20:47:08.525632 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:08.525249    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7592-140704134831680" event={"ID":"e5efe276-6d9d-4f0f-9682-237754cac87c","Type":"ContainerDied","Data":"3ef056728f4642a5bcbc078d2787fdecb8de6bc0cfca542346ec5b4214673736"}
Jul 15 20:47:09.675897 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:09.675871    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:09.701820 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:09.701790    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7592-140704134831680_e5efe276-6d9d-4f0f-9682-237754cac87c/test-curl-7592-140704134831680/0.log"
Jul 15 20:47:09.726335 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:09.726301    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:09.728765 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:09.728742    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7592-140704134831680"]
Jul 15 20:47:09.848265 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:09.848176    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxn6d\" (UniqueName: \"kubernetes.io/projected/e5efe276-6d9d-4f0f-9682-237754cac87c-kube-api-access-sxn6d\") pod \"e5efe276-6d9d-4f0f-9682-237754cac87c\" (UID: \"e5efe276-6d9d-4f0f-9682-237754cac87c\") "
Jul 15 20:47:09.850238 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:09.850204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5efe276-6d9d-4f0f-9682-237754cac87c-kube-api-access-sxn6d" (OuterVolumeSpecName: "kube-api-access-sxn6d") pod "e5efe276-6d9d-4f0f-9682-237754cac87c" (UID: "e5efe276-6d9d-4f0f-9682-237754cac87c"). InnerVolumeSpecName "kube-api-access-sxn6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:47:09.949179 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:09.949142    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxn6d\" (UniqueName: \"kubernetes.io/projected/e5efe276-6d9d-4f0f-9682-237754cac87c-kube-api-access-sxn6d\") on node \"ip-10-0-130-52.ec2.internal\" DevicePath \"\""
Jul 15 20:47:10.536260 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:10.536226    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="361cc178c82d996c8091e3f4b3b2a8794507a89c382d1c5b0e1a95642646b3ee"
Jul 15 20:47:10.536260 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:10.536229    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:10.545831 ip-10-0-130-52 kubenswrapper[2562]: E0715 20:47:10.545801    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7592-140704134831680\" is forbidden: User \"system:node:ip-10-0-130-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-52.ec2.internal' and this object" podUID="e5efe276-6d9d-4f0f-9682-237754cac87c" pod="opendatahub/test-curl-7592-140704134831680"
Jul 15 20:47:11.079154 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:47:11.079118    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5efe276-6d9d-4f0f-9682-237754cac87c" path="/var/lib/kubelet/pods/e5efe276-6d9d-4f0f-9682-237754cac87c/volumes"
Jul 15 20:48:07.964480 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:07.964402    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86d98d5447-db7qh_924198f2-21e9-49fa-8310-87cdfa824e2d/manager/0.log"
Jul 15 20:48:08.330249 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:08.330172    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-84g4h_8bd82a0c-ed6b-4558-8d04-d93485554634/copy-manifests/0.log"
Jul 15 20:48:08.343118 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:08.343093    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-84g4h_8bd82a0c-ed6b-4558-8d04-d93485554634/manager/0.log"
Jul 15 20:48:08.965451 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:08.965407    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54f47c8df8-h92lb_855acac0-fd56-45f7-8660-9e330c20f48e/maas-api/0.log"
Jul 15 20:48:12.060302 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:12.060260    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pm8g5_04e28bed-0d43-4518-8b33-a15c8ef6ec92/manager/0.log"
Jul 15 20:48:12.171168 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:12.171140    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5grkm_694bb4c4-eed2-4908-93a2-b7dd01a579ff/manager/0.log"
Jul 15 20:48:13.340893 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:13.340866    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcf7fc97d-2dfbs_1f51d7bd-6452-4eb9-a03a-211be39ca5dc/kube-auth-proxy/0.log"
Jul 15 20:48:13.699384 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:13.699359    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6vg7p_c5606231-994f-4b2b-9c01-4d7bd6b5d0a2/payload-pre-processing/0.log"
Jul 15 20:48:14.392915 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:14.392889    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-4s5zn_fd7c479c-6664-464e-a5d0-b9f3ca29459d/main/0.log"
Jul 15 20:48:14.616720 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:14.616694    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfcdfh_ca8a4d34-1a12-48e7-b39b-443bd1b53bba/main/0.log"
Jul 15 20:48:14.731275 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:14.731249    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-n45v7_100badf6-fba8-4032-b26f-64b1b6ad9d0f/main/0.log"
Jul 15 20:48:23.412328 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:23.412300    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zh2mc_41d3dc94-0044-4326-92be-d1550a0502b1/global-pull-secret-syncer/0.log"
Jul 15 20:48:23.509568 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:23.509536    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bbmcl_6f6563d9-f278-4f12-a251-c7808bbe1f85/konnectivity-agent/0.log"
Jul 15 20:48:23.564724 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:23.564703    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-52.ec2.internal_a3d8921e521c473a7998a09a95a39b65/haproxy/0.log"
Jul 15 20:48:29.152553 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:29.152529    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pm8g5_04e28bed-0d43-4518-8b33-a15c8ef6ec92/manager/0.log"
Jul 15 20:48:29.177219 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:29.177190    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5grkm_694bb4c4-eed2-4908-93a2-b7dd01a579ff/manager/0.log"
Jul 15 20:48:31.530175 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.530142    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4p974_1dbe4d63-7e0b-4e0c-af84-2dd518d0b103/kube-state-metrics/0.log"
Jul 15 20:48:31.548916 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.548888    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4p974_1dbe4d63-7e0b-4e0c-af84-2dd518d0b103/kube-rbac-proxy-main/0.log"
Jul 15 20:48:31.575469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.575445    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4p974_1dbe4d63-7e0b-4e0c-af84-2dd518d0b103/kube-rbac-proxy-self/0.log"
Jul 15 20:48:31.603672 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.603653    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78f57cd896-2j8tk_6c6aa1e8-4483-4e2b-acaa-e18b0340d3f6/metrics-server/0.log"
Jul 15 20:48:31.731901 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.731872    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d7m9l_bb58ad33-fa45-4f75-9aee-1e8262cb1c35/node-exporter/0.log"
Jul 15 20:48:31.753895 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.753874    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d7m9l_bb58ad33-fa45-4f75-9aee-1e8262cb1c35/kube-rbac-proxy/0.log"
Jul 15 20:48:31.776027 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.776006    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d7m9l_bb58ad33-fa45-4f75-9aee-1e8262cb1c35/init-textfile/0.log"
Jul 15 20:48:31.950159 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.950129    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5770e7f4-305b-48b5-86ae-5361439494a5/prometheus/0.log"
Jul 15 20:48:31.976935 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.976912    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5770e7f4-305b-48b5-86ae-5361439494a5/config-reloader/0.log"
Jul 15 20:48:31.997261 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:31.997237    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5770e7f4-305b-48b5-86ae-5361439494a5/thanos-sidecar/0.log"
Jul 15 20:48:32.020041 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.020018    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5770e7f4-305b-48b5-86ae-5361439494a5/kube-rbac-proxy-web/0.log"
Jul 15 20:48:32.041651 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.041623    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5770e7f4-305b-48b5-86ae-5361439494a5/kube-rbac-proxy/0.log"
Jul 15 20:48:32.063668 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.063646    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5770e7f4-305b-48b5-86ae-5361439494a5/kube-rbac-proxy-thanos/0.log"
Jul 15 20:48:32.092821 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.092802    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5770e7f4-305b-48b5-86ae-5361439494a5/init-config-reloader/0.log"
Jul 15 20:48:32.117911 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.117893    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jpbvn_126a7c91-08a6-4a2b-8da0-6ac52e50b17d/prometheus-operator/0.log"
Jul 15 20:48:32.138490 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.138470    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jpbvn_126a7c91-08a6-4a2b-8da0-6ac52e50b17d/kube-rbac-proxy/0.log"
Jul 15 20:48:32.160940 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.160921    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-ncpf6_b984a64f-db50-4da4-8200-9b8d334ee230/prometheus-operator-admission-webhook/0.log"
Jul 15 20:48:32.268652 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.268594    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66c9f7d96d-5nh7v_2ce6a864-c87e-4dbb-817c-dedc1c940431/thanos-query/0.log"
Jul 15 20:48:32.290813 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.290792    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66c9f7d96d-5nh7v_2ce6a864-c87e-4dbb-817c-dedc1c940431/kube-rbac-proxy-web/0.log"
Jul 15 20:48:32.315709 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.315693    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66c9f7d96d-5nh7v_2ce6a864-c87e-4dbb-817c-dedc1c940431/kube-rbac-proxy/0.log"
Jul 15 20:48:32.336604 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.336584    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66c9f7d96d-5nh7v_2ce6a864-c87e-4dbb-817c-dedc1c940431/prom-label-proxy/0.log"
Jul 15 20:48:32.356189 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.356170    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66c9f7d96d-5nh7v_2ce6a864-c87e-4dbb-817c-dedc1c940431/kube-rbac-proxy-rules/0.log"
Jul 15 20:48:32.377284 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:32.377267    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66c9f7d96d-5nh7v_2ce6a864-c87e-4dbb-817c-dedc1c940431/kube-rbac-proxy-metrics/0.log"
Jul 15 20:48:33.499622 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.499560    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"]
Jul 15 20:48:33.500197 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.500178    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5efe276-6d9d-4f0f-9682-237754cac87c" containerName="test-curl-7592-140704134831680"
Jul 15 20:48:33.500313 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.500201    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5efe276-6d9d-4f0f-9682-237754cac87c" containerName="test-curl-7592-140704134831680"
Jul 15 20:48:33.500313 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.500302    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5efe276-6d9d-4f0f-9682-237754cac87c" containerName="test-curl-7592-140704134831680"
Jul 15 20:48:33.507069 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.504757    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.508278 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.508255    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhp28\"/\"kube-root-ca.crt\""
Jul 15 20:48:33.508396 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.508361    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dhp28\"/\"default-dockercfg-kjwc9\""
Jul 15 20:48:33.509083 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.509065    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhp28\"/\"openshift-service-ca.crt\""
Jul 15 20:48:33.512936 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.512919    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-9szdc_52a14153-48da-4f0d-8f04-f7ea004b250b/networking-console-plugin/0.log"
Jul 15 20:48:33.513287 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.513266    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"]
Jul 15 20:48:33.534870 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.534847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-podres\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.535001 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.534886    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g5dp\" (UniqueName: \"kubernetes.io/projected/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-kube-api-access-4g5dp\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.535001 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.534915    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-sys\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.535001 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.534987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-lib-modules\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.535128 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.535026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-proc\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636166 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-sys\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636321 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-lib-modules\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636321 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-proc\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636321 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636262    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-podres\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636321 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-sys\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636321 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g5dp\" (UniqueName: \"kubernetes.io/projected/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-kube-api-access-4g5dp\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636345    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-lib-modules\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636370    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-podres\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.636495 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.636389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-proc\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.644845 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.644823    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g5dp\" (UniqueName: \"kubernetes.io/projected/ac14a170-472a-4ae3-b7cf-4d39d97fbee6-kube-api-access-4g5dp\") pod \"perf-node-gather-daemonset-krk2r\" (UID: \"ac14a170-472a-4ae3-b7cf-4d39d97fbee6\") " pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.819196 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.819118    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:33.948422 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:33.948304    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"]
Jul 15 20:48:33.951114 ip-10-0-130-52 kubenswrapper[2562]: W0715 20:48:33.951087    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podac14a170_472a_4ae3_b7cf_4d39d97fbee6.slice/crio-7b8be47229e735a54665ef0a876c7e92d424c7b77bd41ef6103aa1db0457a11a WatchSource:0}: Error finding container 7b8be47229e735a54665ef0a876c7e92d424c7b77bd41ef6103aa1db0457a11a: Status 404 returned error can't find the container with id 7b8be47229e735a54665ef0a876c7e92d424c7b77bd41ef6103aa1db0457a11a
Jul 15 20:48:34.673171 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:34.673143    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-6srt7_8e001282-41de-470d-b1ff-88e93afe680a/download-server/0.log"
Jul 15 20:48:34.938649 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:34.938619    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r" event={"ID":"ac14a170-472a-4ae3-b7cf-4d39d97fbee6","Type":"ContainerStarted","Data":"57d8535501a72858053ad33a35e0d0e2ab7eefc3b65de65e74d8f6f6ca961ca9"}
Jul 15 20:48:34.938842 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:34.938674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r" event={"ID":"ac14a170-472a-4ae3-b7cf-4d39d97fbee6","Type":"ContainerStarted","Data":"7b8be47229e735a54665ef0a876c7e92d424c7b77bd41ef6103aa1db0457a11a"}
Jul 15 20:48:34.938842 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:34.938722    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:34.963469 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:34.963423    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r" podStartSLOduration=1.963410965 podStartE2EDuration="1.963410965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:48:34.962364298 +0000 UTC m=+2996.410737394" watchObservedRunningTime="2026-07-15 20:48:34.963410965 +0000 UTC m=+2996.411784100"
Jul 15 20:48:36.244750 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:36.244716    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-54fvs_3bd9fdcd-774e-495b-888f-ddfe727138bf/dns/0.log"
Jul 15 20:48:36.265369 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:36.265343    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-54fvs_3bd9fdcd-774e-495b-888f-ddfe727138bf/kube-rbac-proxy/0.log"
Jul 15 20:48:36.389404 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:36.389379    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwq7z_d2a9b2f0-1e6d-44d2-819f-3e09690c879f/dns-node-resolver/0.log"
Jul 15 20:48:36.958679 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:36.958638    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5867879996-sjtv7_8604be60-3471-4606-8e13-46f3470b44df/registry/0.log"
Jul 15 20:48:37.003061 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:37.002992    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8z8br_27fda661-b8a4-4bed-b977-ccf99ecc406e/node-ca/0.log"
Jul 15 20:48:38.161188 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:38.161162    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcf7fc97d-2dfbs_1f51d7bd-6452-4eb9-a03a-211be39ca5dc/kube-auth-proxy/0.log"
Jul 15 20:48:38.319666 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:38.319628    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6vg7p_c5606231-994f-4b2b-9c01-4d7bd6b5d0a2/payload-pre-processing/0.log"
Jul 15 20:48:39.130158 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:39.130129    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xsqkj_a756270b-4ef2-4628-a1b4-eed8f5b25d3f/serve-healthcheck-canary/0.log"
Jul 15 20:48:39.636254 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:39.636227    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2vxzc_f3d2f159-5f80-4228-bedd-8d261a02e1f4/kube-rbac-proxy/0.log"
Jul 15 20:48:39.662366 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:39.662343    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2vxzc_f3d2f159-5f80-4228-bedd-8d261a02e1f4/exporter/0.log"
Jul 15 20:48:39.689142 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:39.689115    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2vxzc_f3d2f159-5f80-4228-bedd-8d261a02e1f4/extractor/0.log"
Jul 15 20:48:40.953963 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:40.953936    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dhp28/perf-node-gather-daemonset-krk2r"
Jul 15 20:48:42.555020 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:42.554991    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86d98d5447-db7qh_924198f2-21e9-49fa-8310-87cdfa824e2d/manager/0.log"
Jul 15 20:48:42.761777 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:42.761747    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-84g4h_8bd82a0c-ed6b-4558-8d04-d93485554634/manager/0.log"
Jul 15 20:48:42.781357 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:42.781333    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f6d85d69-84g4h_8bd82a0c-ed6b-4558-8d04-d93485554634/copy-manifests/0.log"
Jul 15 20:48:44.211329 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:44.211301    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcf7fc97d-2dfbs_1f51d7bd-6452-4eb9-a03a-211be39ca5dc/kube-auth-proxy/0.log"
Jul 15 20:48:44.255029 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:44.254996    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67d9958876-sdtzq_4af4574e-f8dc-4eb9-8b66-6f16d0fb9f62/manager/0.log"
Jul 15 20:48:44.329346 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:44.329318    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6vg7p_c5606231-994f-4b2b-9c01-4d7bd6b5d0a2/payload-pre-processing/0.log"
Jul 15 20:48:45.798419 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:45.798388    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcf7fc97d-2dfbs_1f51d7bd-6452-4eb9-a03a-211be39ca5dc/kube-auth-proxy/0.log"
Jul 15 20:48:45.921909 ip-10-0-130-52 kubenswrapper[2562]: I0715 20:48:45.921877    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6vg7p_c5606231-994f-4b2b-9c01-4d7bd6b5d0a2/payload-pre-processing/0.log"